run402-mcp

by kychee-com

Overview Schema Related Servers Score Discussions

TypeScript

Local

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v4.17.8
Disambiguation2/5
With 198 tools, many have overlapping or confusingly similar purposes. For example, status, service_status, tier_status, and get_operator_status all sound similar despite different scopes, and a large number of deploy_*, project_*, and notification_* tools create high ambiguity. The detailed descriptions help, but the sheer volume and close names make misselection likely.
Naming Consistency2/5
Naming conventions are inconsistent: some tools use verb_first (list_projects, create_org), others use noun_first (assets_put, jobs_submit, domains_get), and there are one-off names like init, status, and whoami. Prefixes like mailbox_ and contract_ are not used uniformly across related tools (e.g., create_mailbox vs list_mailboxes vs get_mailbox). This mixed style reduces predictability.
Tool Count1/5
198 tools is an extreme mismatch for any MCP server scope, far exceeding the typical 3-15 well-scoped tools. Even for a comprehensive platform like Run402, this count is overwhelming and impractical for an agent to navigate effectively. The set could be consolidated or split into focused servers.
Completeness4/5
The tool set is remarkably comprehensive, covering project lifecycle, deployments, functions, storage, auth, email, billing, notifications, and more. There are minor gaps, such as no direct tool to list/delete project auth users and no organization deletion operation, but the overall surface covers the domain thoroughly.
Average 3.9/5 across 197 of 198 tools scored. Lowest: 2.6/5.
See the Tool Scores section below for per-tool breakdowns.
- 6 of 7 community issues answered or closed in the last 6 months
- 489 commits in the last 12 weeks
- Last stable release on August 12, 2026
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI status not available
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
This repository includes a glama.json configuration file.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility for behavioral disclosure. The description claims 'list sent emails' but the direction parameter can filter inbound or outbound, making the statement misleading. It fails to mention that results are paginated, ordered, or how the API handles limits. This inaccuracy reduces trust and transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise at two sentences but lacks structure (e.g., no bullet points or clear separation of purpose from fields). It is not overly verbose, but the misleading 'sent emails' phrase costs efficiency. Could be slightly improved without adding length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters, no output schema, and no annotations, the description is incomplete. It does not explain pagination behavior (after cursor, limit), default direction, or how to handle the mailbox parameter when the project has one mailbox. The return format is only hinted ('Shows message ID, template...') without details on order or structure. More context is needed for reliable agent usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, so the parameters are already well-documented. The tool description does not add any additional meaning or context about the parameters beyond what the schema provides. According to guidelines, baseline 3 is appropriate when schema coverage is high.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists emails from a mailbox and enumerates returned fields (message ID, template, recipient, status, timestamp). It implicitly distinguishes from sibling tools like get_email (single email) and send_email. However, it inaccurately specifies 'sent emails' when the direction parameter allows both inbound and outbound, which slightly undermines clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., get_email for individual messages, list_mailboxes for mailbox metadata). Missing context on pagination (after cursor, limit), default behavior, or filtering with the direction parameter. The description does not help the agent decide when to include or omit optional parameters.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must fully convey behavioral traits. It discloses that the tool can create/update and optionally send an invite, but does not describe authorization details beyond 'service key', potential destructive effects (e.g., overwriting existing users), or side effects of the invite. Rate limits and idempotency are also not addressed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 16 words, which is concise and front-loaded with the main action. However, it could be slightly more structured (e.g., separating create/update and invite functionalities) without adding length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 6 parameters and no output schema, the description covers the core actions but misses critical details: the dependency of 'redirect_url' on 'send_invite' (though present in schema), the nature of 'client_state', and the distinction between creating and updating. The sibling overlap also reduces completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description summarizes the tool's capabilities but does not add meaning for individual parameters like 'client_state' or 'redirect_url'. It reiterates 'send_invite' and 'is_admin' but without additional context beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description states 'Create or update a project auth user', clearly indicating the verb and resource. However, it does not differentiate from the sibling tool 'invite_auth_user', which likely overlaps in functionality (sending invites). This lack of distinction prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'invite_auth_user'. The description mentions 'with the service key' but does not explicitly state prerequisites or context for choosing this tool over others. No exclusions or when-not-to-use information is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description implies initiating a payment flow but does not disclose side effects, authorization requirements, URL expiration, or idempotency. With no annotations, the description carries full burden but fails to provide necessary transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (two sentences) and front-loaded with purpose. However, it could include slightly more structure without becoming lengthy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is incomplete given the tool's complexity: 6 parameters, no output schema, no annotations. It omits what the tool returns, prerequisites, and error handling, leaving significant gaps for an agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds minimal value by listing product names, but the schema already documents each parameter's meaning and constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a Stripe checkout URL for an organization and lists three products. However, it does not differentiate from sibling billing tools like billing_history or get_quote.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It does not explain prerequisites, when to call this tool, or what not to use it for.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It mentions the passkey requirement for authorization, but does not disclose if deletion is permanent, reversible, or has side effects like cascading deletions. For a destructive operation, more transparency is needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no filler. It is efficiently structured but could benefit from mentioning the expected outcome or return value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple (one parameter, no output schema), but the description does not specify what happens after deletion (e.g., success confirmation, error conditions). Given the simplicity and lack of output schema, this is a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not add meaning beyond the input schema, which already fully describes the 'rule_id' parameter with 100% coverage. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it deletes a Telegram routing rule, but the term 'Telegram routing rule' might confuse agents expecting 'notification rule' based on the tool name. Still, the action and resource are clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description only mentions a prerequisite (operator_passkey assurance) but does not indicate when to use this tool versus alternatives like list_notification_rules or create_notification_rule, nor any context about when deletion is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It indicates a read operation but fails to disclose authentication requirements, error handling, or behavioral details like what happens if the email does not exist or what 'details' entails. Minimal behavioral insight.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no unnecessary words. It is front-loaded with the verb and resource, making it efficient and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema and only three parameters, the description is too sparse. It omits return value format, error conditions, and how replies are included. A more complete description would cover these aspects.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so each parameter is documented in the schema. The tool description adds no extra meaning beyond what the schema already provides. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and the resource ('a sent email with details and any replies'), which is specific and distinguishable from siblings like get_email_raw (raw source) or list_emails (listing all). However, it does not explicitly differentiate itself, so it earns a 4.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, nor does it mention prerequisites or exclusions. The agent receives no help in deciding between this and sibling tools like get_email_raw or list_emails.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description does not disclose behavioral traits such as read-only nature, side effects, or access requirements beyond the cryptic note 'Read via run402 doctor'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short but includes a cryptic instruction ('Read via run402 doctor') that may confuse the agent. Could be more streamlined.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and zero parameters, the description lists content items but lacks details on output structure. Adequate for a simple snapshot tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has zero parameters, so schema coverage is 100%. Baseline score of 3 applies, and description adds no parameter-specific meaning as none exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it provides a compact operator-health snapshot, listing key components (contact assurance, critical items, etc.), and distinguishes from sibling tools like service_health by focusing on operator-specific items.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like service_health or service_status. The description lacks context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It indicates mutability (POST/PATCH/DELETE) but does not disclose authentication requirements, side effects, rate limits, or data persistence details. The key_type parameter is mentioned only in schema, not description.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise (20 words), with purpose front-loaded. Every word earns its place; no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite 6 parameters and no output schema, description is minimal. It lacks examples, response format hints, or usage patterns. An agent would need more detail to use the tool effectively, especially for constructing queries and handling responses.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds the concept of 'query params' but does not elaborate on any parameter beyond the method. It provides no additional semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool queries or mutates data via PostgREST REST API and supports common HTTP methods. However, it does not explicitly differentiate from similar tools like run_sql or contract_read, though the context of REST API vs SQL or contract calls is implied.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. Given siblings like run_sql (direct SQL) and contract_read (smart contracts), an agent would benefit from context on selecting the appropriate database access method.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are absent, so the description must convey behavioral traits. It indicates a write operation ('Update') and a prerequisite (service_key), but does not disclose side effects (e.g., whether changes affect active sessions), idempotency, or error scenarios. The description lacks depth for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences. The first sentence lists the action and fields, the second mentions a requirement. While efficient, it could be better structured with bullet points or separate lines for readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 6 parameters, no output schema, and no annotations, the description should provide more context about expected behavior, return values, and side effects. It only covers the basic action and a single requirement, leaving gaps for an agent to correctly invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds minimal value by listing 4 of 6 parameters and noting a default (allow_password_set default: false), but omits 'allowed_email_domains' and 'project_id'. The description does not compensate for schema completeness.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and the resource 'project auth settings', listing four specific fields. This distinguishes it from sibling tools like create_auth_user which manage users, or set_user_password which is per-user. However, it could be more explicit about the scope (project-level configuration).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions a requirement ('Requires service_key'), which is a basic usage guideline. However, it does not specify when to use this tool versus alternatives, such as when to update individual user settings instead. No exclusions or conditions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden for behavioral traits. It only states 'Delete a published app version' without disclosing side effects, irreversibility, or prerequisites, which is insufficient for a destructive operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The single-sentence description is concise and front-loaded with the action. However, it is perhaps too brief, lacking additional context that could improve usability without adding significant length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema and the destructive nature of the tool, the description does not explain what happens after deletion (e.g., permanent removal, effect on app status). It leaves critical behavioral details unspecified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (both parameters have descriptions). The description adds no extra meaning beyond what the schema already provides, meeting the baseline but not exceeding it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'delete' and the specific resource 'published app version', distinguishing it from siblings like publish_app, update_version, and list_versions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as update_version or unpublish flows. The description merely states the action without context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any behavioral traits such as read-only nature, pagination, sorting, or effects on the system. The agent is left unaware of whether this operation is safe or has side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no wasted words. It is front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one parameter and no output schema, the description is minimal but does not cover expected response format, potential pagination, or ordering. It is adequate but leaves gaps for agent decision-making.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter 'project_id', which is adequately described in the schema. The description does not add any additional meaning beyond the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List published versions of a project' clearly states the action (list) and the resource (published versions) with a specific scope (of a project). It distinguishes itself from sibling tools like 'delete_version' and 'update_version' by focusing on listing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., 'list_projects' or other list operations). The description lacks context on prerequisites, such as requiring a project ID, and does not indicate when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and description lacks details on side effects, required permissions, or behavioral constraints beyond 'publish' and 'set visibility and tags'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence that efficiently conveys the core action and key options. Slightly more detail could be included without harming conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given lack of annotations and output schema, the description should cover default values, effect of 'forkable', and what happens post-publish. It does not, leaving gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% coverage of parameter descriptions, so baseline is 3. Description adds context for visibility and tags but does not elaborate on description or fork_allowed parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'publish' and resource 'project as a forkable app', distinguishing it from sibling tools like fork_app and browse_apps.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like fork_app or browse_apps. Does not mention prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavior. It implies restarting execution and optionally modifying retry settings, but fails to mention side effects (e.g., state changes, resource usage) or prerequisites (e.g., run must be terminal). The description is too brief for safe agent use.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the main action. It wastes no words but could include more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 6 parameters, a nested object, and no output schema, the description is incomplete. It omits details on return values, required state of the run, and potential side effects, which are critical for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 83%, so the baseline is 3. The description adds 'optional retry override and optional wait,' but the schema already describes these parameters adequately. No additional semantic meaning beyond the schema is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (redrive), the resource (failed/cancelled/expired durable function run), and mentions optional parameters (retry override and wait). It effectively distinguishes from sibling tools like create_function_run or cancel_function_run.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide guidance on when to use redrive vs alternatives such as creating a new run or invoking the function again. No exclusions or context on prerequisites are given, leaving the agent without clear usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses that email is sandboxed/off and cron is off unless enabled. However, it does not mention other critical behaviors such as whether the operation is destructive, cost implications, permission requirements, or side effects on the parent project.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, front-loaded with the main purpose. Every word adds value; no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite 6 parameters and no output schema, the description omits what the tool returns (likely a branch identifier or status). It also lacks prerequisites (e.g., parent project must exist) and does not mention that a snapshot may be created if not provided.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description restates defaults (email off/cron off) which are already in schema. It adds no additional meaning beyond what the schema's parameter descriptions provide.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action 'Create' and resource 'contained branch project'. It distinguishes from siblings by mentioning 'from a fresh or existing snapshot' and specifying email/cron containment. However, it doesn't explicitly differentiate from similar creation tools like 'create_project_snapshot'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies usage for creating a contained branch with sandboxed email and disabled cron, but lacks explicit guidance on when NOT to use this tool or when to prefer alternatives. No exclusions or context comparing with sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description mentions releasing CAS references, but without annotations, it fails to disclose key behavioral traits such as permanence of deletion, required permissions, whether it can be undone, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that efficiently conveys the main action. It could be slightly expanded, but it is not verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a simple delete operation with only two parameters, the description is mostly adequate, but it lacks behavioral and usage context that would help an agent decide when and how to invoke it safely.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema coverage, the input schema already describes the parameters clearly. The description adds no additional meaning beyond the schema, so baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete') and the resource ('project data snapshot') and adds detail about releasing CAS references, which distinguishes it from related tools like create, get, or restore.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as restore_project_snapshot or other deletion tools. No prerequisites or conditions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits, but it only states the action. It does not mention that deletion is permanent, that it requires appropriate permissions, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words, achieving maximum conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is too minimal for a destructive tool with no annotations or output schema. It lacks context about irreversibility, permissions, and post-deletion state.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the description adds no extra meaning beyond the schema's parameter descriptions. The baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'delete' and the resource 'secret' in the context of a project, distinguishing it from related tools like set_secret and list_secrets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor are there prerequisites or caveats mentioned. The description is purely functional.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavior. It states that the URL stops serving content, which is a key effect. However, it does not mention whether the subdomain becomes available for reuse, if there are any irreversible consequences, or authorization requirements (e.g., ownership).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with no wasted words. The second sentence adds value by stating the effect. Could be slightly more structured (e.g., indicating return value) but is appropriately concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple deletion tool with no output schema, the description covers purpose and effect. However, it lacks details on return behavior, error conditions, and prerequisites. Given the absence of annotations, more completeness would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Both parameters have descriptions in the schema (100% coverage). The tool description does not add any extra meaning beyond what the schema provides, so it meets the baseline but does not improve understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Release a custom subdomain' clearly states the action and resource. The added detail 'The URL will stop serving content' clarifies the consequence. It distinguishes from siblings like 'claim_subdomain' and 'add_custom_domain', but does not explicitly differentiate from 'remove_custom_domain' (which may have similar semantics).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives such as 'remove_custom_domain' or 'disable_sender_domain_inbound'. The description does not mention prerequisites, side effects, or context for invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavior. It only says 'List all', but does not mention pagination, ordering, filtering, rate limits, or any side effects. This is insufficient for a listing operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that communicates the core purpose with no unnecessary words. It is well front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool lacks an output schema and the description does not clarify the return format or pagination behavior. For a list tool, this leaves important context incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already fully describes the single parameter 'project_id' with a clear description. The tool description adds no additional semantic value beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List'), the resource ('ProjectDomain aggregates'), and the scope ('for a project'). It effectively distinguishes from sibling tools like 'domains_get' or 'domains_activate'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as 'domains_get' (for a single domain) or other listing tools. No prerequisites or exclusions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It implies a read operation by using 'List', but does not disclose that it is non-destructive, what authentication is needed beyond the access token, or any other behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the verb, no redundant or extra words. It is highly concise and structured effectively.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list operation with no output schema, the description should hint at the return format (e.g., list of passkey objects). It does not describe what 'active passkeys' means or the structure of the output, leaving the agent with incomplete information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description does not add any additional meaning beyond the parameter names and descriptions already present in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'the authenticated user's active passkeys', which is specific and distinguishes it from sibling tools like delete_passkey or passkey_register_options.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like delete_passkey or passkey_register_verify. The description only states what it does, not the context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description is the sole source for behavioral traits. It does not disclose pagination, result limit, what 'active' means, or any side effects. Merely states the action without depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence that is front-loaded with the action. Every word contributes meaning, no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with no output schema and no annotations, the description should clarify response format, pagination, and ambiguity of 'active' and 'contained'. It falls short, leaving critical gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear description for project_id. The tool description adds no new meaning beyond what the schema already provides, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'list' and the specific resource 'active contained branch projects for a parent project'. Distinguishes from siblings like create_project_branch and delete_project_branch by focusing on listing active contained branches.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides no guidance on when to use this tool versus alternatives like list_projects or other list tools. No prerequisites, conditions, or exclusions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; the description does not disclose side effects, permissions needed, or response behavior. It only states the basic action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, clear sentence that directly states the tool's purpose with no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of annotations and output schema, the description lacks details on TTL meaning, default behavior for optional ttl_days, and return value, making it incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds no extra meaning beyond the schema, which already includes parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Extend' and identifies the resource as 'contained branch project's TTL', clearly distinguishing it from siblings like create_project_branch or delete_project_branch.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, no prerequisites or exclusions mentioned, leaving the agent to infer context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so description must carry full burden. It only states the revocation action without explaining idempotency, error behavior (e.g., if grant doesn't exist), or side effects. Minimal behavioral insight.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: two sentences, no waste. First sentence states action and parameters, second adds prerequisite. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter revocation tool with no output schema, the description covers the core action, required params, and a key prerequisite. Could mention return value or error conditions, but completeness is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. The description adds minimal value by mentioning the example `grt_...` for grant_id. Baseline 3 is appropriate as schema already provides parameter details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states action ('Revoke a per-project capability grant by id') and lists required parameters. However, it does not differentiate from related siblings like 'create_project_grant' or clarify the broader context of grants.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Only prerequisite ('Requires you to be an owner') is provided. No guidance on when to use this tool versus alternatives, such as when to revoke vs. create a grant.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so description must disclose safety. It states 'returns results as a markdown table' but does not mention side effects of DDL, permissions needed, rate limits, or error behavior. Critical details for a tool that executes arbitrary SQL.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with action, no redundancy. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, and the description only states 'returns results as a markdown table' without detail on error handling, pagination, transaction behavior, or response structure. Incomplete for a SQL execution tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds no parameter-specific details beyond the schema; it does not elaborate on the 'sql', 'params', or 'project_id' parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verb 'Execute SQL' and resource 'provisioned project', clearly distinguishing from sibling tools like 'rest_query'. It specifies DDL and queries, and mentions markdown table output.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like 'rest_query'. No when-not-to-use or prerequisite information provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full transparency burden. It only states cost and auth requirements, but fails to disclose behavioral traits like idempotency, uniqueness constraints, or consequences of claiming an already-claimed subdomain.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences, front-loading the core action and example. Every word earns its place, with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given moderate complexity (3 params, no output schema), the description covers purpose, cost, and auth. However, it lacks post-condition details (e.g., DNS propagation) and error handling context, leaving gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents each parameter. The description adds the example subdomain format and cost/auth note, but does not clarify the optional project_id role or provide further semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Claim a custom subdomain') and the resource ('point it at an existing deployment'), with an example ('myapp.run402.com'). It distinguishes from sibling tools like add_custom_domain by emphasizing 'custom subdomain'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions 'Free, requires service_key auth' but provides no guidance on when to use this tool versus alternatives (e.g., add_custom_domain, delete_subdomain). No explicit when-to-use or when-not-to-use context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description lacks details on irreversibility, dependencies, or confirmation steps. 'Delete' implies a destructive action but does not confirm whether the operation is permanent or if it can be undone.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with 7 words, front-loaded with the verb and resource. No extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete operation, the description is minimally adequate but lacks context on return values, error conditions, and side effects. Without annotations, more completeness would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the description adds no extra meaning beyond the already clear parameter names and types. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete') and the resource ('a deployed function from a project'), distinguishing it from sibling tools like delete_mailbox or delete_project.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., deactivate vs delete), no prerequisites or conditions mentioned. Among many function-related tools, the description does not help an agent decide when deletion is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description mentions 'purge its resources' implying irreversibility, but lacks details on side effects, permissions, or whether it cascades. Transparency is insufficient for a deletion tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, 8 words, no fluff. Efficient and direct.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema. Lacks mention of return values, irreversibility confirmation, or prerequisites. For a simple delete operation, it is moderately complete but could be more transparent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%. The description does not add significant meaning beyond the schema's parameter descriptions; it provides minimal added value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Delete') and resource ('branch project'), clearly distinguishing from sibling tools like 'create_project_branch' and 'list_project_branches'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool, prerequisites, or alternatives. For a destructive operation, such context is missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It reveals a key behavioral trait (returning mailbox addresses to managed fallback) but omits details like destructiveness, permissions, or what happens to the domain itself.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, well-structured sentence that conveys the core action without extraneous words. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has simple parameters and no output schema. The description covers the basic action but fails to explain return values, 'managed fallback', or side effects. It is adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema coverage is 100% and both parameters are described in the schema. The description adds no additional meaning beyond what the schema provides, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Disconnect' and the resource 'ProjectDomain', and distinguishes the action from sibling tools like domains_activate or domains_check. However, it does not explicitly differentiate from all siblings, so it's a 4.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, nor any prerequisites or exclusions. It only states the action without context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavioral traits. It only states a read operation but omits details on authorization requirements, rate limits, or side effects. The brevity leaves ambiguity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no extraneous words, front-loaded with the key action. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description hints at the return value ('data snapshot and its next actions') but lacks detail on structure. For a simple read tool with 2 params, it is adequate but not rich.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for both parameters (project_id, snapshot_id). The description adds no extra meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'one project data snapshot and its next actions', distinguishing it from sibling tools that create, delete, list, or restore snapshots.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., list_project_snapshots, restore_project_snapshot). No context about prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits, but it only states the basic action. It does not mention side effects, reversibility, permissions, or error conditions, which is insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, direct sentence with no extraneous information. It is perfectly concise and front-loaded with the key action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple mutation tool with 2 parameters and no output schema, the description is adequate but minimal. It lacks context about job state transitions or consequences of cancellation, so it is not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already describes both parameters. The description adds no additional meaning beyond the schema, meeting the baseline but providing no extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the action ('Cancel') and the resource ('queued or running managed job'), clearly distinguishing it from sibling tools like jobs_submit or jobs_get.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., jobs_purge, jobs_get) or when not to use it. The description provides no context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits like read-only nature, rate limits, or authorization needs. It only states the action without additional transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise with one sentence plus a brief param note. No unnecessary words, and important information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the tool is simple, the description does not explain return structure, pagination, or role details. With no output schema, additional context would be beneficial for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides full coverage for the single parameter (org_id). The description redundantly mentions 'Params: `org_id`' but adds no new meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and the resource 'members of an org and their roles', distinguishing it from sibling tools like 'list_orgs' and 'get_org'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives such as 'add_org_member' or 'set_org_member_role'. The description lacks context about prerequisites or limitations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses that the tool stores the passkey (write operation), but doesn't elaborate on side effects like permanent storage or authentication requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded, no wasted words. Could benefit from a second sentence for context, but still concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema or annotations, and 5 parameters. The minimal description leaves out necessary context like the flow from passkey_register_options and the expected format of the response parameter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% so baseline is 3. The description adds no additional meaning beyond the schema's parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'verify' and 'store' and the resources 'browser WebAuthn registration response' and 'user's passkey', distinguishing it from siblings like passkey_register_options and passkey_login_verify.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives. The description does not mention the prerequisite call to passkey_register_options or that it follows the browser registration step.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description does not disclose side effects (e.g., overwriting previous pointer, validation failure behavior, persistence) beyond the basic action. The burden is high without annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is efficient and front-loaded, containing key action and result. No redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and minimal description, the tool lacks contextual details about return values, error handling, and side effects, making it incomplete for a simple yet state-changing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and description adds no extra meaning beyond the schema's parameter description. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool validates a project and sets it as the active pointer, using specific verbs and resource. It distinguishes itself from siblings as no other tool sets the active project pointer.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the action of setting active project, but no explicit guidance on when to use vs alternatives or prerequisites is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavioral traits. It does not mention idempotency, side effects (e.g., immediate POST notifications), authentication needed, or whether the webhook secret is returned. The description only states it receives notifications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loaded with the core purpose, and each sentence adds necessary information. No redundancy or wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters and no output schema, the description covers the basic purpose and events but lacks information on webhook lifecycle (e.g., how to manage, what happens after registration). It is adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds no extra meaning beyond what's in the schema, such as constraints like HTTPS for url or that mailbox can be omitted when only one mailbox exists. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'register' and resource 'webhook on mailbox', and lists specific email events (delivery, bounced, complained, reply_received). This distinguishes it from sibling tools like delete_mailbox_webhook.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., update_mailbox_webhook, list_mailbox_webhooks). There is no mention of prerequisites like ensuring the mailbox exists or that the URL is reachable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses the need for a service key and active add-on, and mentions optional parameters. However, it lacks detail on error behavior, rate limits, or return format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences covering purpose, requirements, and options. No extraneous text, clear and front-loaded. Slightly over-simplified but effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description omits return value details. It also does not mention character limits (though present in schema) or error cases. It covers core purpose but not enough for complete agent guidance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are already documented. The description adds minimal value by noting 'optional source language and context hint,' which is redundant with schema descriptions. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Translate text to a target language') and the resource (text). It is unique among siblings, with no other translation tool, so no ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It only mentions prerequisites (service key, add-on) but does not specify contexts where translation is appropriate or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden for behavioral disclosure. It mentions delegation to the SDK and a result envelope, but does not disclose side effects, auth requirements, or what 'plan only' entails (the dry_run parameter covers that in schema, but not in the main description). Lack of transparency on mutation behavior is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences (55 words), front-loading the core action and following with delegation details. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 13 parameters and no output schema or annotations, the description provides a reasonable high-level overview but omits details on SDK return structure, step lifecycle, and how parameters like dry_run affect behavior. Adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so each parameter has a description. The main description adds value by explaining the high-level workflow and the return envelope ('graph steps, resources, diagnostics, next_actions'), which compensates for the lack of an output schema. Baseline 3 plus extra context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool plans or runs the app-aware workflow, specifying the resource ('run402 up') and source ('local path or repo URL'). It distinguishes from siblings by naming the canonical workflow, though it could explicitly differentiate from related tools like 'deploy'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for planning or running the up workflow, but does not provide explicit when-to-use or when-not-to-use guidance, nor does it suggest alternatives among the many sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It only states the function (browse) but does not specify read-only nature, pagination, or any side effects. The description lacks transparency about what the tool does beyond the action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, concise and front-loaded with the core action. No unnecessary words or information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one optional parameter and no output schema, the description covers the essential purpose and filtering capability. It could mention that a list of apps is returned, but is otherwise adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers 100% of the parameter (tags) with its own description. The tool description does not add additional meaning beyond the schema, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action ('browse public apps') and the specific purpose (available for forking), with optional filtering by tags. This distinguishes it from sibling tools like 'fork_app' or 'get_app'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. No mention of scenarios where browsing is appropriate or when other tools (e.g., 'get_app' for a specific app) should be used instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Discloses source (CloudWatch) and log types, but does not mention limitations like latency, max tail, pagination, or read-only nature beyond the verb 'get'. Lacks depth for a tool with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short, clear sentences with no unnecessary words. Front-loaded verb and resource. Efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description covers return content. Lacks details on return format (e.g., fields like timestamp, message), but sufficient for a simple read operation with well-documented parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with adequate field descriptions. The tool description adds no parameter information, but the schema already serves its purpose. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb (get) and resource (recent logs from deployed function) and specifies content (console.log/error output, stack traces). However, it does not differentiate from sibling get_function_run_logs, which may cause confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like get_function_run_logs. No mention of prerequisites, context, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only states it fetches logs but does not disclose any behavioral traits like read-only nature, rate limits, pagination, or what happens if no logs exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence. It is front-loaded and has no fluff, though it could benefit from a brief mention of context (e.g., requires project_id and run_id).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description does not explain return format. Schema covers parameters adequately. For a log-fetching tool, the description is minimally complete but lacks output details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description need not add much. It does not provide additional meaning beyond the schema, but the schema itself fully describes parameters. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Fetch' and resource 'logs correlated to one durable function run', clearly distinguishing it from sibling tools like 'get_function_logs' which might return all logs, and 'list_function_runs' which lists runs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies that this tool is for logs of a specific run, but does not explicitly state when to use it over alternatives such as 'get_function_logs' or 'get_function_run'. No guidance on context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It states it's a report (read), but does not disclose safety, auth requirements, side effects, or error handling. For a tool with no annotations, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with dash-delimited list of report contents. Efficiently communicates scope without redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description lists key report elements (API calls, storage, limits, lease expiry), providing adequate context for a simple read operation. Minor gap: no mention of read-only nature.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a single parameter (project_id) described as 'The project ID'. Description adds no further meaning beyond the schema, so baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Get project usage report' with specific resource enumeration (API calls, storage, limits, lease expiry), distinguishing it from sibling tools like ai_usage.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives such as billing_history or ai_usage. The description implies a straightforward read but does not provide context for excluding other tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses that the tool creates/updates and sends an invite requiring service_key and redirect_url. However, it omits details about return values, side effects, or what happens on update vs create. The mention of 'service_key' not present in the schema also causes confusion.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, consisting of two short sentences. It front-loads the main action. However, it could benefit from a slightly more structured format to separate the action from requirements.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (5 parameters, 3 required) and no output schema, the description should explain what the tool returns or side effects. It only states it sends an invite link but does not specify the return value (e.g., success confirmation, user object). This leaves a significant gap for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, but the description adds minimal additional meaning. It mentions 'requires service_key' which is not in the schema, contradicting the parameter set. It does clarify that 'redirect_url' must be an allowed one and 'is_admin' sets project_admin status, but overall adds little value and introduces confusion.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates/updates a project auth user and sends a trusted invite magic link. This distinguishes it from siblings like 'create_auth_user' (which likely just creates without invite) and 'request_magic_link' (which is for existing users).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions required prerequisites ('service_key' and 'allowed redirect_url'), providing basic usage context. However, it does not explicitly state when to use this tool vs alternatives, nor does it provide when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses return behavior (credentials on success, payment details if x402 needed), which is helpful. However, it lacks details on destructive nature, authorization requirements, or idempotency, and annotations are absent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and to the point, but could be slightly expanded to include key contextual details without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (optional parameters, tier options, payment requirement), the description is somewhat incomplete. It does not mention the tier choices or org_id implications, nor does it have an output schema to supplement.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema documentation coverage is 100%, so the description adds no extra parameter info beyond the schema. No per-parameter elaboration is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Provision') and the resource ('a new Postgres database'), distinguishing it from sibling tools like 'provision_signer' or project management tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as when to provision a signer or use a different project creation flow. The mention of 'cold-start path' is only in the schema, not in the description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses destructive action and side effect, but no annotations are present. Lacks details on permanence, permissions, or error handling for missing keys.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no redundant information, front-loaded action and side effect.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequately states action and side effect but lacks behavior on missing keys or error states. For a deletion tool, more context on reversibility would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema already fully describes both parameters with 100% coverage. Description adds no additional semantic value beyond what is in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'delete a blob' and specifies the side effect 'decrement storage_bytes'. It distinguishes from sibling asset tools (assets_get, assets_ls, assets_put, assets_sign) by indicating a destructive operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, no prerequisites or conditions for deletion provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the full burden of behavioral disclosure. It mentions that the wallet is resolved to its organization over SIWX and that a wallet not linked requires an admin key, which is useful. However, it does not disclose whether the operation is read-only (implied by 'View'), whether results are paginated (despite the 'limit' parameter suggesting pagination), or what the response structure contains.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the purpose in the first sentence. The second sentence adds essential behavioral context without waste. Every part earns its place, making it highly concise and structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with no output schema, the description covers purpose and key behavioral context (resolution, admin key). However, it omits details about the returned data structure (e.g., what fields are in the billing history) and does not mention that results may be paginated despite the 'limit' parameter. Given the simplicity, it is adequate but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers both parameters with descriptions, achieving 100% coverage. The tool's description does not add new semantic meaning beyond the schema; it only provides context about wallet resolution. Per guidelines, baseline is 3 when schema coverage is high, so this score is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'View billing ledger history' with the specific resource 'agent's allowance wallet'. This gives a clear verb and resource. However, it does not explicitly differentiate from similar sibling tools like 'allowance_status' or 'check_balance', which could lead to confusion about when to use this tool over others.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides some usage context, such as wallet resolution over SIWX and the need for an admin key if the wallet is not linked. However, it lacks explicit guidance on when to use this tool versus alternatives like 'allowance_status' or 'get_usage'. No alternatives are named, leaving the agent to infer the correct usage scenario.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses idempotency and email sending. With no annotations, the description carries full burden. It doesn't mention failure modes (invalid email, Stripe issues) or confirm that creation is non-destructive. Adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states purpose and constraints, second adds behavioral traits. No redundant words, front-loads key info.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, idempotency, email verification. Lacks response format, error handling, and whether operation is synchronous. Adequate for a simple creation tool but missing typical context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter 'email' with schema description already covering constraints. The tool description restates 'Stripe-only, no wallet' but does not add new parameter-related meaning beyond the schema. With 100% schema coverage, baseline is 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'Create' and resource 'email-based organization'. Adds constraints 'Stripe-only, no wallet required', which helps distinguish from wallet-based org creation. However, it does not explicitly differentiate from sibling tool 'create_org', which may be more general.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context that it sends a verification email and is idempotent. Implies usage for email-based, Stripe-only organizations, but no explicit guidance on when to use this versus alternatives like 'create_org' or 'link_wallet_to_organization'. No when-not-to-use statements.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must disclose behavioral traits. It mentions manual capture and internal use, but does not detail effects, permissions, side effects, or any destructive vs non-destructive nature.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, front-loaded with the core action, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, but description gives basic info. However, it lacks details on what the snapshot contains, how long it takes, or what the response includes. Adequate for a simple creation tool but could be more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a single parameter 'project_id' described as 'Project ID to snapshot.' Description adds no additional meaning beyond schema, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states action ('Capture') and resource ('manual project data snapshot'), distinguishing it from siblings like get_project_snapshot, delete_project_snapshot, restore_project_snapshot.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description provides context that snapshots are internal restore points, not downloadable archives, but does not explicitly state when to use this tool versus alternatives or mention any prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only states 'Delete' but does not disclose whether the action is irreversible, what happens to related data, or if authentication is required (though access_token is in schema). Minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with verb and resource. No unnecessary words or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple deletion tool with 3 parameters and no output schema, the description is adequate but lacks details on return values, error cases, or side effects. It is minimally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and schema descriptions are adequate. The description does not add additional meaning beyond 'by id' for the passkey_id parameter. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete') and the resource ('one authenticated-user passkey by id'). It distinguishes from sibling tools like list_passkeys and create_auth_user.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (delete a passkey by id) but does not provide explicit guidance on when not to use it, prerequisites, or alternatives among the many sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, and the description does not disclose any behavioral traits such as read-only nature, permissions required, or side effects. The agent is left to infer that this is a simple fetch operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that immediately conveys the action and resource. No unnecessary words, highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (2 required parameters, no output schema), the description is minimally adequate but could include more details such as the return value format or that both parameters are required. It does not fully equip the agent for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with both parameters (run_id, project_id) described. The description adds context about fetching by fnrun_ id, aligning with the run_id parameter, but does not provide additional meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Fetch one durable function run by fnrun_ id.' Clearly specifies the action (fetch) and the resource (a single durable function run), distinguishing it from sibling tools like list_function_runs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance is provided. The name implies fetching a specific run, but no alternatives are mentioned. Usage is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description bears full burden. It only states 'Get', which implies a read operation, but fails to mention permissions, rate limits, or what the response contains. Minimal behavioral insight.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One clear, concise sentence with no extraneous information. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval tool with full schema coverage, the description is adequate but minimal. It does not mention that no output schema exists, nor does it clarify the scope of the job run data returned.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both parameters described. The description adds no additional meaning beyond the schema, meeting the baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'managed job run by id', which is specific and distinguishes from sibling tools like 'jobs_cancel', 'jobs_logs', etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like 'jobs_logs' or 'jobs_download_artifact'. The description implies usage for retrieving a specific job run, but lacks when-not-to-use context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden for behavioral traits. It only mentions the listing and filterability but omits whether the operation is read-only, any side effects, pagination behavior (cursor/limit details), rate limits, or authentication needs. The description provides minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently conveys the tool's purpose and key filterable dimensions. Every word earns its place with no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and 8 parameters, the description provides a basic overview but does not explain return structure, default sorting, or ordering. It lacks depth about what the agent can expect after invocation, leaving some gaps for a moderately complex listing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description lists some filter options (status, event_type, time window, limit, cursor) but does not add meaning beyond what the schema already provides. It serves as a high-level summary without additional detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists 'durable runs for a function', which is a specific resource. It distinguishes from siblings like 'get_function_run' (single run) and 'list_functions' (list functions). The mention of filterable options adds specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing runs but does not explicitly state when to use this tool versus alternatives like 'get_function_run' or 'cancel_function_run'. There is no guidance on prerequisites or when not to use it, so the usage context is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden but discloses no behavioral traits beyond 'list all'. No mention of pagination, ordering, or read-only nature.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no redundancy, front-loaded with action and resource. Highly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple list operation, but lacks details about response content (e.g., webhook properties) and the optional mailbox parameter usage. No output schema to fill gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already describes parameters. Description adds no extra meaning beyond restating the resource context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly specifies action (list), resource (webhooks), and scope (on the project's mailbox), distinguishing it from siblings like list_mailboxes and list_mailbox_webhook_deliveries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like list_mailbox_webhook_deliveries or filtering options. Usage is implied by the resource name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of disclosing behavioral traits. It reveals that the tool returns delivered, failed, and skipped attempts and supports pagination and filtering. However, it does not explicitly state that the operation is read-only, mention rate limits, or describe the response format. The addition of the audit log context provides moderate value beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that efficiently conveys the tool's purpose and key features (audit log, status types, pagination, filters). No superfluous words or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of an output schema and annotations, the description provides the core purpose and capabilities but omits details about the response structure, error handling, or any side effects. For a data retrieval tool with four parameters, the description is minimally adequate but leaves important context unspecified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with all four parameters having descriptions. The description's mention of filtering by event type or since timestamp mirrors the parameter descriptions without adding new semantic details. Thus, the description does not significantly enhance understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists the operator's notification audit log, specifying three types of attempts (delivered, failed, skipped). This distinguishes it from sibling tools like get_notification_preferences (which retrieves preferences) and test_notification (which sends a test).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It mentions pagination and filtering capabilities but does not address scenarios where a different tool (e.g., get_notification_preferences) would be more appropriate. Given the large set of sibling tools, explicit usage guidelines are absent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavior. It mentions optional filtering and pagination, but does not specify ordering, default page size, or confirm that the operation is read-only. The schema provides parameter details, but behavioral traits like side effects are missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads key information. No redundant or unnecessary text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters and no output schema, the description covers the basic functionality but lacks details on pagination mechanics, default behavior, and ordering. It is adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, providing full parameter documentation. The description adds minimal value beyond the schema by reiterating 'optional kind filter and keyset pagination'. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List'), the resource ('project data snapshots'), and key features ('optional kind filter and keyset pagination'). This distinguishes it from sibling tools like 'create_project_snapshot' or 'get_project_snapshot'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied ('list snapshots'), but there is no explicit guidance on when to use this tool versus alternatives (e.g., 'get_project_snapshot' for a single snapshot). No when-not or exclusion criteria are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
While the description indicates a safe read operation, it lacks details on pagination, limits, or what 'claimed' means relative to sibling tools. No annotations are provided.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no unnecessary words, making it easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description omits what the response contains (list of strings? objects?), and lacks context on how this tool relates to claim_subdomain and delete_subdomain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter (project_id) has full schema coverage, and the description adds no additional meaning beyond the schema's 'The project ID'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (list), resource (subdomains), and scope (claimed by a project), distinguishing it from sibling tools like list_custom_domains.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as list_custom_domains or claim_subdomain, and no exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits (e.g., that this generates options without side effects, or requires user interaction). It only states the action without explaining the result or implications for the agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with key information. No redundant or extraneous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description is minimal. It covers the basic purpose but lacks details on what the tool returns or any prerequisites, making it adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All three parameters are described in the schema (100% coverage), so the description adds no new meaning beyond 'create for a project app origin'. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create') and the resource ('WebAuthn passkey login options') with specific scope ('for a project app origin'). It distinguishes from sibling tools like 'passkey_register_options' by specifying 'login'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives like 'passkey_register_options' or 'passkey_login_verify'. Usage is implied by the name and description, but no direct instructions or exclusions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses the basic action but does not explain side effects, return value format, or whether it creates a session. Missing behavioral details like idempotency or expiration.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words, directly communicates the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple tool, but lacks mention of return format and the next step (verification with passkey_register_verify). No output schema or annotations to supplement.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all 3 parameters with descriptions. The description adds the context 'for the authenticated user' but does not explain parameter roles beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool creates WebAuthn passkey registration options for the authenticated user, specifying the verb, resource, and user scope. It distinguishes from sibling tools like passkey_register_verify and passkey_login_options.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool (for passkey registration) but lacks explicit guidance on when not to use it or comparisons to alternatives like passkey_register_verify or passkey_login_options.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It does mention the required service_key for authorization, but lacks details on reversibility, idempotency, or what happens if the user already has the role. The extra sentence about admin secrets is tangential.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, but the second sentence about admin secrets is only tangentially related and adds unnecessary length. The purpose is front-loaded, but the structure could be tighter without the digression.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and moderate complexity, the description covers the basic purpose and a required authorization. However, it omits important context such as required preconditions (user must exist in the project), potential errors, and output format. It is adequate but has clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds minimal value beyond the schema: it implies the 'email' parameter is the user's email address and that 'project_id' is a project identifier. No additional constraints or formats are given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Promote a user to project_admin role' with the specific verb 'promote' and resource 'user to project_admin role'. It distinguishes from the sibling tool 'demote_user' which performs the opposite action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions a prerequisite 'Requires service_key' but lacks explicit guidance on when to use this tool versus alternatives like 'demote_user' or 'set_org_member_role'. The usage context is implied but not clearly delineated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the full burden. It discloses that certain operations require specific assurances (email_verified, operator_passkey), which is behavioral. However, it does not mention error behaviors, idempotency, or side effects beyond the constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. It efficiently communicates the tool's purpose and key behavioral constraints.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no output schema, and the description does not mention return values. For an update tool with 8 optional parameters, the description is adequate but could be more complete by indicating success/failure indicators.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with detailed parameter descriptions. The tool description adds no significant additional meaning beyond repeating the webhook_url assurance already in the schema. Baseline score applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update operator notification preferences' using a specific verb and resource. It distinguishes from the sibling 'get_notification_preferences' by implying a write operation. However, it does not explicitly differentiate from other update tools in the sibling list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides important context on assurance requirements for specific changes (cross-wallet effects, webhook URL). However, it does not state when to use this tool versus alternatives or explicitly indicate when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It describes output content but discloses no behavioral traits (e.g., read-only nature, required permissions, rate limits, side effects). Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with key purpose. Every word adds value. No wasted text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter tool lacking output schema, description adequately lists output components. Could mention structured nature or use cases, but sufficient for basic understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Zero parameters with 100% schema coverage. Description adds no parameter info (unnecessary), no deduction needed. Baseline 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it provides a 'full organization snapshot' listing key components like allowance, billing, tier, projects. Distinguishes from siblings like allowance_status or service_status which are more specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies use when needing a comprehensive overview ('single-call overview'), but does not explicitly state when not to use or suggest alternatives like allowance_status or tier_status for specific queries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavioral traits. It indicates mutation ('update') but lacks details on prerequisites (e.g., must the version be published?), side effects, or permissions. The description lists updatable fields but does not go beyond that.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence that front-loads the verb and resource, lists the target fields clearly. No unnecessary words or repetition. Excellent conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the basic purpose and parameters, but lacks information about return values (no output schema), behavior on failure, or any constraints (e.g., version must be published). For a 6-parameter update tool, this is adequate but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 6 parameters are described in the input schema (100% coverage), so the schema already provides meaning. The description only repeats the field names without adding extra value like formatting constraints or interdependencies. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (update), the resource (metadata of a published app version), and specifies the fields that can be updated (description, tags, visibility, fork_allowed). This distinguishes it from sibling tools like delete_version or publish_app.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The description implies it is for updating a version's metadata, but does not mention when not to use it or suggest alternatives. Given sibling names, the purpose is clear but lacks explicit context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It notes that canonical SDK errors are preserved and implies a read-only operation by stating 'Fetch'. However, it does not explicitly declare safety, rate limits, or authentication requirements, making it adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three front-loaded sentences: purpose, return list, usage tip, and error behavior. No fluff, every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description explicitly lists all returned fields (metadata, state kind, inventories, etc.) and explains site_limit usage. Error handling is mentioned. This covers most necessary context for a fetch tool, though some format details are omitted.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters with descriptions, setting a baseline of 3. The description adds extra context for site_limit ('Use site_limit to cap large site inventories') but does not enhance the meaning of project_id or release_id beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Fetch a release inventory by id', using a specific verb and resource. It lists the returned fields, distinguishing it from sibling tools like deploy_list (which lists releases) and deploy_release_diff (which compares). No tautology.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as deploy_list or deploy_release_active. The description only offers a usage tip for site_limit but lacks comparative context or when-not-to-use advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description lists the return fields (lifecycle state, block number, gas used, etc.), which adds behavioral insight. However, it does not explicitly disclose that this is a read-only operation, nor does it mention any side effects or prerequisites. Given no annotations, more transparency would be beneficial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It front-loads the action and lists return values efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple lookup tool with two parameters, the description adequately covers purpose and return values. However, the lack of output schema is compensated by listing return fields. Minor gap: no mention of error structure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both parameters well-described. The description adds 'by call_id' but does not provide additional meaning beyond the schema. This meets the baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Look up a previously submitted contract call by call_id,' providing a specific verb and resource. This distinguishes it from sibling tools like contract_call or contract_read, which have different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given on when to use this tool versus alternatives. The description simply states what it does without providing context for appropriate usage or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description lacks disclosure of prerequisites, side effects, error conditions, or security implications. Does not mention that it likely requires a prior challenge or that it creates a session.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, front-loaded sentence with 13 words. No redundant information or extraneous details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool has low complexity with 3 well-described parameters and no output schema. Description covers the essential purpose, though additional context about the prerequisite challenge_id could be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for each parameter. Description adds no extra meaning beyond the schema, such as format or constraints for the 'response' parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb (verify), resource (WebAuthn assertion), and outcome (return auth session). Distinguishes from siblings like passkey_register_verify and passkey_login_options.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance. Implies usage in login flow after passkey_login_options, but no alternatives or exclusions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses auto-detection and potential payment requirement, but fails to mention side effects, error handling, or irreversible actions. With no annotations, more detail is expected for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences with front-loaded purpose. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequately describes behavior and output for a single-parameter tool without output schema. Missing error cases or preconditions, but sufficient for most use cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and parameter description already mirrors tool behavior. Description adds no new semantic information beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it subscribes, renews, or upgrades tier, with auto-detection based on allowance state. This distinguishes it from read-only sibling 'tier_status', though no explicit differentiation is given.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage when changing tier, but no explicit guidance on when to use versus alternatives like 'tier_status' or 'allowance_create'. Lacks exclusion criteria or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that it 'does not expose the challenge secret', which is a useful behavioral note. However, with no annotations, it fails to mention other important traits such as idempotency, side effects, rate limits, or authentication requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—two short sentences that front-load the action and add one key behavioral fact. Every word earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and no output schema, the description is minimally adequate. However, it lacks information about return values, success/failure indicators, or what happens after the challenge is sent, which an agent would need for reliable invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, and the schema coverage is 100%. The description doesn't need to add parameter details, and its concise note about the challenge secret adds value beyond the empty schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Start or resend') and the resource ('operator email reply challenge for the active agent contact email'). It is specific and informative, but does not differentiate from sibling tools like 'get_agent_contact_status' or 'set_agent_contact'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for initiating or re-sending an email challenge for agent contact verification, but provides no explicit guidance on when to use this tool versus alternatives, nor any preconditions or scenarios where it should not be used.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description covers non-idempotency, conflict/cooldown/limit errors, and conditional return of settings. However, it does not disclose authentication requirements or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences, front-loaded with purpose, no filler. Every sentence provides meaningful information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 2 required parameters and no output schema, the description covers return values, error behavior, and address format. It is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds value by explaining the resulting email address format and the allowed cross-project slug matching, beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a project-scoped mailbox local part and provides the address format. It distinguishes from siblings like delete_mailbox or get_mailbox by implication but lacks explicit differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives context (project-scoped, slug unique within project) but does not provide explicit when-to-use or alternatives among sibling mailbox tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavior. It states the role change and a requirement, but does not mention side effects, idempotency, reversibility, or error states. Adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence covering action, target, effect, and requirement. Very concise and front-loaded. Could be improved by splitting into two sentences for readability, but no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple mutation tool with no output schema, the description covers the essential: what it does, the new role, and a key requirement. Missing post-conditions or relationship to promote_user, but still sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds no extra meaning beyond the schema descriptions (email and project_id are already described). No format, constraints, or examples provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Demote a user from project_admin role by email'), specifies the resource (user by email), and the outcome ('Reverts to default authenticated role'). It distinguishes from sibling tools like promote_user. This is specific and actionable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions a prerequisite ('Requires service_key') but does not provide explicit guidance on when to use this tool versus alternatives like promote_user or remove_org_member. It lacks when-not-to-use context or scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It extensively covers behavioral details: CAS transfer, idempotency with idempotency_key, warning handling (allow_warnings, stop before upload), return values (release_id, URLs, warnings, logs), and secret management (must be set separately). This is highly transparent for a complex tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is verbose and reads like a specification. While it is front-loaded with 'Unified apply primitive', it includes many detailed clauses that could be streamlined for faster parsing. Some sentences are dense and could be broken down.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (13 parameters, nested objects), the description covers most aspects: migrations, routing, secrets, assets, site, subdomains, and return values. There is no output schema, but the description lists return values. No major gaps are apparent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 69%, and the description adds significant meaning beyond the schema. It explains concepts like migration id vs name semantics, public_paths modes (explicit/implicit), route target types (function/static), and the secrets require/delete pattern. This adds value for an agent understanding parameter usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it is a 'Unified apply primitive' and lists the components it handles (database, secrets, functions, etc.). However, it does not explicitly distinguish itself from sibling tools like deploy_function or deploy_site, which would help an agent choose correctly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides detailed semantics (e.g., replace vs patch, public_paths modes) but lacks explicit guidance on when to use this tool vs alternatives like deploy_function or deploy_site. The context implies it's for complex multi-resource deployments, but no direct comparison is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions that the source project and plan stay untouched, which is good. However, it does not clarify what happens to the contained branch (e.g., is it created, left behind?) or any authorization needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loading the action and including a key behavioral note. Every part is essential and there is no unnecessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description could elaborate on the rehearsal report format or prerequisites. It covers the core action and safety but misses context on return value and lifecycle of the contained branch. Adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description does not add new meaning beyond the parameter descriptions; it restates concepts like 'persisted apply plan' and 'contained branch' without elaboration on teardown options or project_id usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it runs a persisted apply plan against a contained branch and returns the rehearsal report, with a specific verb and resource. It also distinguishes from sibling tools like 'deploy' by emphasizing rehearsal and no changes to source.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
While the description implies testing before actual deployment, it does not explicitly state when to use this tool versus alternatives like 'deploy' or 'deploy_resume'. No when-not-to-use or alternative names are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses the return report contents and the polling behavior of wait. However, it does not mention side effects, authentication requirements (beyond what schema provides), or rate limits, leaving some gaps for a read-heavy verification tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences efficiently deliver the purpose, return value, and a key usage hint. No filler or redundancy, and the most critical information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no output schema, the description partially compensates by listing report components. However, it lacks details on return structure, error handling, or prerequisites beyond what schema provides. It is adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description adds minimal extra meaning beyond the schema definitions. The only slight addition is clarifying that `timeout_seconds` applies when `wait` is true, but that is already implied by schema. The description does not significantly enhance parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (verify) and resource (gateway/edge release coherence). It distinguishes this tool from siblings like deploy, deploy_rehearse, and deploy_diagnose_url by its specific purpose of verifying coherence and returning a detailed report.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies this tool is used after a deploy to check coherence, but it does not explicitly compare to alternatives like deploy_release_active or deploy_diagnose_url. It provides some guidance on the wait parameter but lacks clear when-to-use and when-not-to-use context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must bear full disclosure. It mentions 're-observe' implying a side effect (triggering DNS check), but does not clarify if it is read-only, idempotent, or what state changes occur. It also omits details on response format, pagination, or latency for a check tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single 14-word sentence, front-loading the core action and purpose. Every word adds value; no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has simple parameters (2 required strings) and no output schema or annotations. The description covers basic function and usage context but fails to describe return format or potential side effects. For a check tool, this is adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% description coverage for both parameters (domain, project_id). The description does not add extra semantic meaning beyond what the schema already provides, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's verb ('Re-observe DNS/provider state', 'return') and resource ('ProjectDomain'). It distinguishes from sibling tools like domains_get by emphasizing it triggers a new observation, not just retrieval. The phrase 'preflight and drift detector' sets it apart from modification tools like domains_activate or domains_apply.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly recommends use 'as a preflight and drift detector before validation cycles', providing clear context. It does not list alternatives or exclusions, but the purpose is clear enough to guide selection among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Given no annotations, the description should disclose behavioral traits. It mentions returning checks, DNS records, and next actions, indicating verification steps. However, it does not mention potential destructiveness, authentication needs, or rate limits, leaving significant gaps for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no filler. First sentence states purpose, second states output. Slightly verbose but efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 3 parameters and no output schema, the description covers purpose, scope, and return value. It is mostly complete but could mention prerequisites or when to use this tool vs other domain siblings.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds context by listing aspects (web, email, etc.) but mostly repeats schema field descriptions. It does not provide additional parameter semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates or updates a project-scoped ProjectDomain desired state for web, email sending, inbound receive, mailbox addresses, and activation. This verb+resource combination distinguishes it from sibling tools like domains_activate or domains_check.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for setting or updating domain configuration but does not explicitly state when to use this over alternatives like domains_activate or domains_apply. No exclusions or comparison to siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses automatic verification before import, dry_run and require_runnable support, and SECRET_VALUES_REQUIRED reporting. With no annotations, the description provides moderate behavioral context but omits details on error handling, permissions, or side effects like cleanup on failure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no waste. Purpose is front-loaded, followed by key features. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers main behavioral aspects (auto-verification, dry_run, require_runnable, secret reporting) but does not describe the return value format or error conditions. For a complex tool with 7 parameters and no output schema, more detail on response behavior would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are well-documented there. The description adds minimal extra meaning beyond mentioning dry_run and require_runnable as features, which is already in the schema. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb (import), resource (verified portable archive), and destination (new local Run402 Core project). Distinguishes from sibling tools like verify_project_archive and inspect_project_archive by indicating it performs import with automatic verification.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage contexts through mentions of dry_run (to verify only) and require_runnable (to enforce secret provisioning), but lacks explicit comparison to sibling tools or guidance on when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It declares the tool is 'idempotent, safe to re-run', which is key behavioral information for a tool that performs multiple actions. It lists the actions performed (allowance, faucet, tier status) but does not detail the exact side effects or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the key actions and ends with a clear behavioral note. Every word earns its place; there is no fluff. It is optimally concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool combines multiple actions (allowance setup, faucet request, tier check), the description adequately lists them. However, it lacks details about return values, prerequisites (e.g., authentication, project context), and error handling. For a critical bootstrap tool, more completeness is expected.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one optional parameter 'rail' with a full description and enum values (100% coverage). The description adds no additional meaning to the parameter beyond what the schema provides. Therefore, the score is at baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: it sets up allowance, requests faucet funding, and checks tier status, all in a single call. The verb 'bootstrap' effectively summarizes the resource being set up. However, it does not differentiate from sibling tools like 'allowance_create', 'request_faucet', and 'tier_status', which perform these actions individually.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description labels the tool as a 'single-call bootstrap' and notes it is 'idempotent, safe to re-run'. This implies usage for initial setup and indicates it can be run multiple times without harm. However, it does not explicitly state when to use this tool versus calling the individual tools, nor does it mention any prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility. It discloses that active runners are terminated, which is valuable. However, it does not mention irreversibility, required permissions, or potential impacts on other resources. More detail would be beneficial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence. It communicates the core action and a key behavioral detail without any fluff or repetition. Highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (one parameter, no output schema), the description is fairly complete. However, it lacks information about return values, error handling, or any prerequisites. For a destructive action, more context would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 100% coverage for the single parameter project_id with a description 'The project ID'. The tool description adds no additional semantic meaning beyond what the schema already provides. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verb 'purge', identifies the resource 'managed job runs', and scopes it to 'for a project'. It also mentions a unique behavioral aspect: 'terminating known active runners first'. This clearly distinguishes it from sibling tools like jobs_cancel or jobs_submit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
While the description implies this tool is for bulk cleanup of job runs, it does not explicitly state when to use this tool over alternatives like jobs_cancel. There is no mention of prerequisites, limitations, or exclusions. The guidance is minimal.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses at-least-once delivery, retries, backoff, and the dead-letter queue concept. However, it omits pagination behavior, rate limits, auth requirements, and whether the operation is read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (3 sentences), front-loaded with the purpose, and each sentence adds value without redundancy. It efficiently covers the core concept and critical instructions.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (5 params, no output schema, no annotations), the description covers essential behavioral aspects but lacks details on output shape (e.g., fields beyond idempotency_key), pagination usage of 'after', and the 'limit' parameter cap. It is adequate but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds value by mentioning the need to dedupe on idempotency_key and to filter by status, but does not significantly enhance understanding of individual parameters beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists webhook delivery rows for the project's mailbox, using a specific verb and resource. It distinguishes from sibling tools like list_mailbox_webhooks and includes behavioral context (durable, at-least-once retries, dead-letter queue).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description suggests using this tool to inspect lost events by filtering on status, but it does not explicitly state when to use versus alternatives (e.g., redrive_mailbox_webhook_delivery). No when-not-to-use guidance is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose all behavioral traits. It only states that values and hashes are never shown, but omits other important behaviors such as output format, pagination, error handling, or authorization requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, consisting of two sentences with no unnecessary words. The purpose is front-loaded, and every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the basic purpose and a key behavioral constraint, but lacks details about the output structure, potential errors, or pagination. For a simple list tool, it is adequate but has clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides 100% coverage with a description for the single parameter 'project_id'. The description does not add any additional semantic information beyond what is in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List secret keys for a project' with a specific verb and resource, and differentiates from sibling tools like set_secret and delete_secret by noting that values and hashes are never shown.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'use this only to check which keys are configured', providing clear context for when to use the tool. However, it does not mention when not to use it or provide alternative tool names.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Mentions 'loss statement' and 'atomic restore' but does not clearly state that the current project state will be destroyed or require confirmation details. Lacks full disclosure of consequences.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with core purpose, no redundant words. Highly efficient for a restore tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers basic usage but lacks details on return values, required permissions, or what 'loss statement' means. Adequate but with notable gaps for a destructive operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline 3. Description adds context for confirm parameter (dry vs execute) but does not elaborate beyond what schema provides for other params.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it plans or confirms a project snapshot restore, distinguishing two modes. It differentiates from sibling tools like create/delete/list snapshots.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when to omit or pass the confirm parameter for dry plan vs execution. No mention of when not to use, but the two-mode guidance suffices.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description discloses that email alerts fire on threshold breach, but lacks details on side effects, idempotency, permissions, or error conditions. Adds some context but not full behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, clear and front-loaded. No unnecessary words. Every sentence provides value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema and no annotations. Description lacks return value, error scenarios, prerequisites. For a tool with 3 required parameters, more context is needed for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters with descriptions. The description adds context that threshold_wei triggers email alerts but does not enhance parameter meaning beyond schema. Baseline score is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (set threshold), the resource (KMS signer), and the context (email alerts when balance drops). It distinguishes from siblings like check_balance or set_auto_recharge.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for monitoring signer balances but does not explicitly state when to use versus alternatives like check_balance or set_auto_recharge. No when-not or context for alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description bears full responsibility. It discloses that the tool is free and requires a service key, but does not describe whether it is read-only, rate limits, or other behavioral traits. The return value description is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences that front-load the purpose and then add practical info (free, service key). No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (2 required params, no output schema), the description covers purpose, return values, pricing, and auth. Lacks detail on moderation categories or text limits, but is largely complete for its complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions already provided for both parameters. The tool description adds no additional parameter-level meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose ('Run content moderation on text') and specifies output ('Returns flagged status and category scores'). It distinguishes from sibling tools like ai_translate by focusing on moderation rather than translation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for checking text content but does not provide explicit guidance on when to use vs. alternatives or when not to use. No exclusions or conditions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full burden. It discloses the operation is a read ('Get') and the scope, but does not mention authorization needs, rate limits, idempotency, or any side effects. Minimal transparency beyond the basic purpose.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence front-loads the verb and resource, efficiently conveying purpose and output. No fluff or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with one parameter and no output schema, the description covers purpose, scope, and key output fields. It does not detail return format or pagination, but that is likely unnecessary for a quota query. The lack of annotations is partially mitigated by the clear description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage for the only parameter (project_id) is 100% with a description. The tool description adds value by clarifying the output (used words, quota, remaining balance) and temporal scope (current billing period), which are not in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get AI translation usage' with specific outputs: used words, quota, remaining balance. It distinguishes from siblings like 'get_usage' and 'allowance_status' by specifying AI translation context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions 'for the current billing period' but does not provide explicit guidance on when to use this tool vs alternatives like 'get_usage' or 'allowance_status'. It lacks when-not-to-use or exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavioral traits. It only states 'Check', implying a read operation, but lacks details on side effects, auth requirements, error scenarios, or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one sentence with 10 words, front-loading the purpose and key details. No extraneous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description explains three key aspects of the return. It is adequate for a simple check tool, though it omits potential output format or error handling.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
This tool has zero parameters, so baseline is 4. The description adds meaning by specifying what aspects are checked (address, network, funding status), which is useful beyond the empty schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the specific verb 'Check' and resource 'local agent allowance status', listing three aspects: address, network, and funding status. It clearly distinguishes from sibling tools like allowance_create and allowance_export.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when needing allowance status, but provides no explicit when-to-use, when-not-to-use, or alternatives. No guidance on prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions listing, prefix filtering, and pagination, but does not disclose other behavioral traits like whether it is read-only, auth requirements, or rate limits. The description is adequate but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two concise sentences with no wasted words. It front-loads the main purpose and efficiently communicates key features.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has a complex input schema (nested filter object) and no output schema. The description does not explain the response format, such as the structure of returned blobs or the pagination cursor details. This leaves significant gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, so the schema already documents each parameter. The description adds value by linking 'optional prefix filter' to the prefix parameter and 'pagination via cursor' to cursor/limit, helping to understand their roles.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists blobs in a project with optional prefix filtering and pagination. It uses specific verbs ('List') and resources ('blobs'), and the context distinguishes it from sibling tools like assets_get, assets_put, and assets_rm.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains what the tool does but does not provide explicit guidance on when to use it versus alternatives, nor does it mention when not to use it. It implies use for listing blobs, but lacks comparative context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
States the precondition (run not completed) but lacks info on side effects, required permissions, idempotency, or what happens after cancellation. No annotations provided to supplement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with key action and condition. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple cancellation action. Covers purpose and precondition. Could be enhanced with return value or error conditions, but not required given no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both parameters documented. Description adds no additional meaning beyond the schema definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'cancel', the resource 'scheduled/queued durable function run', and a condition 'when it has not completed yet'. Distinguishes from sibling tools like 'redrive_function_run' or 'create_function_run'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage when the function run is scheduled/queued and not completed. However, it does not explicitly state when not to use it or mention alternative tools for completed runs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behaviors: writing, encoding via viem, signing via AWS KMS, broadcasting, idempotency, and cost. With no annotations, it carries the full burden. However, it does not describe failure modes, return value (likely a call_id), or prerequisites for successful execution.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, no redundant information, front-loaded with action and resource. Every sentence contributes purpose, process, or cost.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite high parameter count and complexity (KMS signing, gas costs, idempotency), the description is minimal. It omits return value (call_id?), error handling, prerequisites (signer existence, gas balance), and input format for idempotency_key. An output schema or richer description would be needed for full completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% description coverage, so baseline is 3. The description adds value by explaining idempotency_key behavior and cost, but does not further clarify complex parameters like args or abi_fragment beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action ('Submit a smart-contract write call'), the source ('from a KMS signer'), and differentiates from sibling tools like contract_read (read) and contract_deploy (deploy new contract) by specifying it's a write call on an existing contract.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Mentions idempotency via idempotency_key and cost structure, which guides when to use and cost implications. Lacks explicit comparison to sibling tools like contract_read vs. contract_deploy, and does not state prerequisites (e.g., signer must be provisioned, chain gas needed).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden of behavioral disclosure. It states the tool creates a full copy and optionally claims a subdomain, which implies non-destructive behavior, but it does not mention auth requirements, rate limits, or consequences if the subdomain is taken.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loaded with the core action, and contains no redundant information. Every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 3 parameters and no output schema, the description covers the main purpose and key aspects. It could mention error cases (e.g., invalid version_id) or the result (e.g., new project ID), but overall it provides sufficient context for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with each parameter described. The description adds minimal extra meaning beyond the schema, such as clarifying subdomain is optional, but this is already evident from the 'required' list. No deeper insights into parameter values or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool's action ('Fork a published app into a new project') and specifies what it includes (database, functions, site, optionally a subdomain). This distinguishes it from siblings like 'publish_app' or 'delete_project'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage (creating a copy of a published app) but does not provide explicit when-to-use or when-not-to-use guidance. No mention of alternatives like 'import_project_archive' or 'export_project_archive' that might serve similar purposes in different contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must cover behavioral aspects. It correctly indicates a read-only operation but lacks details on authentication, rate limits, or what happens if no contact is set. The returned states are listed, which helps, but more context would be beneficial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that immediately conveys the purpose and possible outputs. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the trivial complexity (no parameters, no output schema), the description adequately tells an agent what to expect. It could mention the subject 'current agent' but the meaning is implicit. Missing some context about when to use, but acceptable for a simple getter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, so the schema provides full coverage. The description adds no parameter info, which is appropriate. A baseline of 4 is justified for zero-parameter tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get'), the resource ('agent contact assurance state'), and explicitly lists all possible return values. It is specific and distinguishes itself from sibling tools that modify or verify contact.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like set_agent_contact or verify_agent_contact_email. The description does not indicate prerequisites or context for invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears the burden. It indicates a read operation ('get details') but does not mention authentication requirements, error handling (e.g., webhook not found), or rate limits. Basic transparency is present but leaves gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence that conveys the essential purpose without any filler. Every word is meaningful and necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval tool without an output schema, the description covers the core functionality adequately. It could optionally mention return fields or error conditions, but completeness is sufficient given the tool's simplicity and the presence of a sibling list tool for context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and each parameter has a description. The description adds no additional meaning beyond what the schema provides. Baseline 3 is appropriate as the schema already documents parameter purposes adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Get details of a specific webhook by ID', which is a clear verb+resource pair. It distinguishes from sibling tools like list_mailbox_webhooks (which lists all) by specifying retrieval by ID.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no explicit guidance on when to use this tool versus alternatives (e.g., list_mailbox_webhooks for listing). Usage is implied by 'by ID', but lacks explicit context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should fully disclose behavior. It mentions 'live native-token balance' and 'Chainlink-cached price', indicating read operations and data freshness. However, it does not explicitly state that this is a read-only operation with no side effects, nor does it mention any necessary permissions or potential errors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single efficient sentence that front-loads the core purpose. Every word adds value; there is no fluff or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple getter with two parameters and no output schema, the description adequately covers what the tool returns (metadata, balance, price). It does not mention error cases or what happens if the signer is not found, but overall it is sufficiently complete for its complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with basic parameter descriptions for 'signer_id' and 'project_id'. The tool description does not add extra meaning beyond what the schema already provides, so baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verbs and resources: 'Get a KMS signer's metadata + live native-token balance + USD-micros'. It clearly distinguishes from sibling tools like 'list_signers' (which lists all signers) and 'delete_signer'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for a specific signer (via signer_id) versus list tools, but it does not explicitly state when to use this tool versus alternatives like 'list_signers' for listing or 'provision_signer' for creating. No when-not or exclusions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description bears full responsibility. It correctly implies a read-only operation and notes legacy epoch millisecond acceptance, but omits details on auth requirements, rate limits, return format, or log retention. The behavioral traits disclosed are minimal but not misleading.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, zero wasted words. The purpose is front-loaded, and the parameter guidance is compact yet informative. Every sentence serves a clear purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (4 parameters, no output schema), the description provides adequate context for a read-only log retrieval operation. It explains the optional parameters and constraints. However, it does not describe the output format or any limitations like pagination or log retention period, which would enhance completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with 'description' fields already explaining each parameter. The description adds some context by stating 'use tail to cap entries' and 'since for an ISO-8601 lower bound', but this largely echoes the schema. No new parameter semantics beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Read' and the resource 'recent runner logs for a managed job', making the tool's purpose unambiguous. It distinguishes from siblings like 'jobs_get' and 'get_function_logs' by specifying 'runner logs' rather than job metadata or function logs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises on using optional parameters 'tail' and 'since', but does not provide explicit guidance on when to use this tool versus alternatives (e.g., 'jobs_get' for job status, 'get_function_logs' for function logs). The context is clear but lacks when-not-to-use or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so description carries full burden. It implies read-only behavior by listing fields. It discloses the caveat about runtime not refreshing on redeploy. However, it does not mention authentication, rate limits, or error conditions, so transparency is adequate but not outstanding.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first covers purpose and output details, second adds usage guidance and behavioral caveat. No redundant information, front-loaded, and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with one parameter and no output schema, the description provides a detailed list of return fields and a behavioral note. It lacks mention of pagination or ordering, but these are not critical for basic usage. Overall sufficient for agent understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter project_id, but its description only says 'The project ID'. The tool description does not add any additional meaning beyond the schema. Baseline 3 is appropriate as the description does not compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List all deployed functions for a project' and enumerates specific fields shown. While it does not explicitly differentiate from siblings like deploy_list or list_versions, the level of detail makes its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a specific alternative tool (functions_rebuild) for stale functions and warns that redeploy without source change does not refresh runtime. This gives clear guidance on when to use another tool and a behavioral nuance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It specifies the HTTP GET method and mentions the preview_path for deep-linking, implying a read-only operation. However, it does not explicitly state that no mutations occur, nor does it disclose any side effects or authentication requirements beyond the implicit wallet context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with clear front-loading: first sentence conveys core purpose, second adds a key detail (preview_path) and the API call. No redundancy or unnecessary text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description specifies the returned data includes preview_path, but lacks details about the full response structure or pagination behaviour. Given the tool's simplicity and the presence of pagination parameters, a brief note on how to iterate pages would improve completeness. Still, it covers essential context for an agent to begin using the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (two parameters: after and limit). The description does not add meaning beyond the schema; the schema already describes 'after' as an opaque cursor and 'limit' as page size with default 50. The description's mention of preview_path is unrelated to parameters, so no additional semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists pending project transfers initiated by the authenticated wallet, distinguishing it from incoming transfers and other transfer-related siblings like list_incoming_transfers, initiate_project_transfer, etc. It also mentions the API endpoint and the preview_path field.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for outgoing initiated transfers, but lacks explicit guidance on when to use this tool versus alternatives such as list_incoming_transfers, preview_project_transfer, or accept_project_transfer. Context from sibling names fills the gap, but the description itself does not provide exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses the inclusion of deleted signers, which is a behavioral trait. However, it does not mention permissions, rate limits, or return format, leaving gaps in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no waste. It is front-loaded with the main action. While concise, it could benefit from a brief mention of the return type without losing efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one parameter and no output schema, the description covers the basic purpose and scope. However, it lacks details about the output (e.g., what properties are returned) and any pagination or ordering, which a complete description should include.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a single parameter 'project_id' described in schema. The description adds no additional parameter information beyond restating 'owned by the project', so it provides no extra value over the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'List' and the resource 'KMS signers', specifies scope 'owned by the project, including deleted ones', and distinguishes it from sibling tools like get_signer (specific signer) and delete_signer.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
While not explicit, the description implies when to use this tool (listing all signers, including deleted) versus siblings like get_signer (single signer) or delete_signer (deletion). The inclusion of 'including deleted ones' subtly guides against using it when only active signers are needed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions the worker will attempt delivery again but does not disclose behavior for non-failed deliveries, rate limits, idempotency, or authorization requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with action and purpose. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description adequately explains the core action and when to use. However, it lacks details on edge cases (e.g., non-failed delivery, success indication) and does not clarify the optional mailbox parameter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description does not add any parameter-specific meaning beyond what the schema already provides (e.g., delivery_id source).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (re-queue), resource (dead-lettered webhook delivery), and purpose (worker attempts again). It distinguishes from sibling tools like list_mailbox_webhook_deliveries by specifying the action on failed_permanent deliveries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context: 'Use after fixing the consumer endpoint.' It implies when to use but does not explicitly state when not to or mention alternatives like checking status first.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description's mention of 'offline and deterministic' partially addresses behavioral traits (no network/project interaction). But it does not disclose side effects (e.g., file overwriting, idempotency), output format, or performance characteristics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with zero waste. First sentence states the purpose, second lists inputs and key characteristics. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, and description does not explain what the tool returns (e.g., generated code as string, file creation, console output). For a moderately complex tool with 5 parameters and no output schema, this is a significant gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents parameters. The description only lists inputs briefly ('table, user_col, role_col, roles[], cache_ttl') without adding deeper meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool generates a role-table migration, requireRole gate snippet, and first-operator bootstrap for Run402 function role gates. Verb and resource are specific, and it distinguishes itself from sibling tools which focus on different tasks (e.g., deploy, manage users, etc.).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description mentions 'offline and deterministic (no project or network)', indicating it's safe for local development and has no external dependencies. However, it does not explicitly state when to use vs alternatives or provide usage exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must fully disclose behavior. It indicates the tool is a mutator (enable/disable) and specifies a prerequisite, but does not mention side effects, reversibility, or response behavior. This is adequate for a simple toggle but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, no redundant information, and immediately conveys the core purpose. It is appropriately sized and front-loaded with the action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity (3 parameters, no output schema, no nested objects), the description covers the main aspects: purpose, condition, and a prerequisite. It could optionally mention the return value or confirmation, but the tool's simplicity makes this acceptable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the schema descriptions already explain each parameter. The tool description adds value by linking the threshold to the condition, but does not provide additional semantic context beyond what the schema offers.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Enable or disable') and clearly identifies the resource ('automatic email pack repurchase'). It also mentions the condition ('when credits drop below a threshold'), distinguishing it from sibling tools that manage other billing aspects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states a prerequisite ('Requires a saved Stripe payment method'), which helps the agent know when it can be used, but it does not provide explicit guidance on when to use this tool versus alternatives like set_low_balance_alert or allowance_create.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It mentions setting or clearing values but lacks details on side effects, permissions, rate limits, or reversibility. For a mutation tool, this is insufficient disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences front-load the key action and provide necessary guidance. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 3 parameters, no output schema, and no annotations, the description adequately explains the tool's purpose and usage hints (like calling list_mailboxes first). It lacks details about return values or confirmation, but is sufficient for basic use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the description adds context by explaining the purpose of each parameter and the null option. However, it does not add significant meaning beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool sets default_outbound_mailbox_id and/or auth_sender_mailbox_id for a project, using a specific verb and resource. It distinguishes from sibling tools like list_mailboxes and send_email by explaining the prerequisite and the effect.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells users to use list_mailboxes first to get a mailbox id, and explains the consequence of not setting defaults. Provides clear context for usage, though it does not explicitly list situations where the tool should not be used.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It notes that the link is 'short-lived', but lacks details about authentication needs, side effects (e.g., invalidating previous links), or what happens after sending.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no unnecessary words. It conveys the action, target, and condition efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no parameters and no output schema, the description adequately explains the core function and a precondition. However, it could be more complete by describing what the user receives or next steps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and the schema coverage is 100% (empty). The description adds meaning by explaining the action, which is sufficient given no parameters. Baseline for no parameters is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (email a link), the target (operator passkey enrollment link), and a precondition (email_verified). It effectively distinguishes from sibling tools like delete_passkey or list_passkeys.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions the requirement 'Requires email_verified', which gives usage context. However, it does not explicitly state when to use this tool versus alternatives, nor does it provide exclusions or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses the critical behavioral trait that 'Events is a full replacement, not a merge', which prevents accidental loss of events. With no annotations, the description carries the behavior disclosure burden. It does not mention other aspects like whether URL validation occurs or if changes are reversible, leaving some gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states the action and scope, second adds the key constraint 'At least one field required' and the full-replacement behavior of events. No unnecessary words, perfectly front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers essential information for a straightforward update mutation: what is updated, that events is a full replacement, and that at least one field is required. Missing details like the expected response format (likely success/failure) and prerequisites (e.g., webhook must exist). However, for a simple tool, this is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the input schema already documents all parameters and their meanings. The description reinforces that events is a full replacement, but this is already in the schema ('full replacement'). It adds minimal new meaning beyond the schema, aligning with baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action 'Update a webhook's URL and/or events' identifying the resource (webhook) and specific fields. The 'at least one field required' and 'Events is a full replacement, not a merge' further specify scope. Distinguishes from sibling tools like register_mailbox_webhook (create) and delete_mailbox_webhook (delete).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides the constraint 'At least one field required' which tells the agent that not all fields are mandatory. However, it does not explicitly mention when to use this tool versus alternatives like register_mailbox_webhook for creation or delete_mailbox_webhook for deletion, leaving context implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description covers key behaviors: direct-to-S3, public/private visibility, immutable URLs, size limits (5 TiB vs 1 MB), image variant generation, EXIF policy, and auto-detected content type. However, it does not mention error cases or conflict behavior (e.g., overwrite vs. fail).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single paragraph with multiple details, not front-loaded. It is adequate but could be more concise and structured (e.g., using bullet points for image handling). Some redundancy with schema descriptions.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers many aspects (purpose, size, visibility, immutable, image processing) but omits the return shape (defers to SDK docs), error handling, and conflict semantics. For a complex mutation with 9 parameters, this is a notable gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, baseline 3. The description adds context beyond schema: size limits for local_path vs content, immutable URL semantics, image processing details, and EXIF policy explanation. This adds significant value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Upload a blob (file or inline content) to project storage via direct-to-S3', specifying the verb and resource. It distinguishes from sibling tools like assets_get (download) and assets_ls (list).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus alternatives. It provides details on behavior but lacks guidance on exclusions or context for choosing over other methods.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility. It lists features like idempotency key, scheduling, retry, and expiry, but omits details on default behavior (e.g., whether wait is off by default), side effects (e.g., charging), or execution guarantees (e.g., what constitutes 'durable'). The schema covers parameter semantics well, but the description lacks broader behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three concise sentences. The first sentence states the core action. The second lists key features. The third gives usage guidance. It is front-loaded, efficient, and contains no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the tool's complexity (14 parameters, nested retry object, no output schema), the description covers purpose and usage well but lacks information on return values (run ID?), error scenarios, and default wait behavior. The schema fills some gaps, but the description alone is insufficient for full agent understanding of the tool's lifecycle.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 93% schema description coverage, the baseline is 3. The description mentions several parameter categories (idempotency_key, scheduling, expiry, retry, wait) but does not add new semantics beyond what the schema already provides for each parameter. The schema descriptions are already detailed (e.g., format examples, mutual exclusivity).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a durable function run, with a required idempotency key. It enumerates supported scheduling types (immediate, delayed, run_at), expiry, retry policy, and optional wait, making the purpose specific and distinct from sibling tools like invoke_function or cron.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises using this tool instead of 'ad hoc cron/polling when work must survive retries or be redriven', providing strong usage guidance. However, it does not directly contrast with the sibling invoke_function, which could be used for non-durable synchronous runs, but the context is clear enough for most agents.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full responsibility. It discloses idempotency, key behavior. It does not mention other traits like potential impact on deliveries, but for a simple delete, this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no filler. Front-loaded with the action. Every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is minimal but covers the core purpose. Lacks information about return values or side effects, which given no output schema could be helpful. Adequate for a simple tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with all parameters described. The description adds no extra meaning beyond the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a webhook', which is a specific verb-resource pair. It also notes idempotency, distinguishing it from other webhook operations like register, update, or list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the name and description, but there is no explicit guidance on when to use vs alternatives, nor when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses handler signature, bundled imports, dependency resolution rules (max entries, rejection of native binaries and @run402/functions), and response fields (runtime_version, deps_resolved, warnings). It does not mention authentication requirements, rate limits, or side effects beyond creation/ update, but the disclosed details are extensive and useful.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized for the tool's complexity. It is front-loaded with the core purpose and progressively adds detail. Every sentence contributes value, though slightly longer than minimal. No redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters, no output schema, and no annotations, the description is fairly complete. It covers handler, dependencies, response, and special notes on schedule triggers. It lacks error handling details or failure modes, but the provided information is sufficient for an agent to invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (baseline 3). The description adds significant meaning: for 'deps', it explains bundling mechanics, rejection rules, and limits; for 'code', it provides exact handler signature; for 'schedule', it clarifies cron format and removal; for 'config', it notes default values. This goes well beyond the schema's descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Deploy a serverless function (Node 22) to a project', specifying the verb (deploy), resource (serverless function), and context (Node 22, project). It distinguishes from siblings like list_functions, delete_function, and the generic deploy tool by focusing on function deployment with explicit handler and runtime details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a guideline for schedule/email triggers ('prefer a unified deploy manifest'), but does not explicitly state when to use this tool versus alternative siblings (e.g., update_function, deploy, deploy_site). It lacks directives on when not to use it or comparison with other tools, leaving the agent without clear selection criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description discloses key behavioral traits: only uploads missing bytes, skips .git/node_modules/.DS_Store, rejects symlinks, and issues no PUTs on unchanged trees. It does not cover error handling or permissions, but is fairly transparent for a deployment tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise at four sentences, front-loading the purpose. It avoids redundancy but could be more structured (e.g., listing constraints clearly). No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description explains input, process, and constraints. It covers what happens (incremental uploads, file skipping) but omits return value or deployment URL. Still, it is complete enough for a tool with standard output expectations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with detailed parameter descriptions that largely mirror the tool description. The tool description adds little beyond the schema, such as the transport version and pricing context, but does not significantly enhance parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Deploy a static site from a local directory' and details the process of walking the tree, hashing, and incremental uploads. It distinguishes itself from sibling tools like deploy_site and deploy_function by specifying the local directory source and transport mechanism.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool vs. alternatives. While it implies use for static site deployment from a local path, it lacks guidance on when not to use it or comparisons with other deploy tools like deploy_site.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the cost and the special condition (suspended signers), but does not explain consequences like irreversibility or what happens to the signer after draining. A score of 3 reflects some transparency but missing important behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states the core purpose, second adds context and cost. No fluff, front-loaded, and every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main operation, cost, and when to use it (suspended signers). However, it lacks details on edge cases (e.g., non-suspended signers, invalid addresses) and does not describe return values. For a moderate-complexity tool with no output schema, a 3 is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with basic descriptions. The tool description repeats cost info for destination_address but does not add significant new meaning beyond the schema. Baseline is 3; slight addition keeps it at 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (drain), the resource (KMS signer's entire native-token balance), and the destination address. It also mentions a specific use case (works on suspended signers). This distinguishes it from sibling tools like 'delete_signer' or 'provision_signer'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly mentions that it works on suspended signers (the safety valve), providing context for when to use it. However, it does not specify when not to use it or mention any alternatives, though the sibling list suggests no direct alternatives exist.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description discloses key behaviors (wait, local write) and explicitly states what is never exported (secrets, credentials, etc.), but does not explicitly declare read-only nature or authorization needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that front-load the main purpose and add critical context without unnecessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 9 parameters and no output schema, the description is minimal. It explains the local write path but does not describe return value when output_path is absent, nor detail the 'wait for readiness' mechanism beyond what parameters cover.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline 3. The description does not add significant meaning beyond schema; it mentions wait and output_path but the schema already describes them adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool exports a Runtime slice as a .r402ar archive, specifies it can wait and write locally, and lists what is never exported, distinguishing it from siblings like import/inspect.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for creating portable archives but does not explicitly state when to use this tool versus alternatives like import or inspect, nor provide exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description includes return fields but does not mention idempotency, permissions, or error behavior (e.g., 404 if not found).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with clear structure: what it returns and why to use it.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema; description covers main returned fields but omits error conditions and structure details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%; description adds no extra parameter info beyond the schema (though it lists response fields).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves mailbox info (ID, address, slug, etc.) for a project, distinguishing it from siblings like create_mailbox or list_mailboxes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit use case ('check if a mailbox exists'), but does not mention when to use an alternative like list_mailboxes or exclude misuse.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
In the absence of annotations, the description discloses the GET HTTP method, the API endpoint, and the fact that entries include a preview_path for deep-linking. It also mentions the version requirement. It does not explicitly state that the operation is read-only, but the nature of listing transfers implies it. Overall, it provides sufficient behavioral context for a list tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—two sentences covering purpose, scope, version, preview_path, and the HTTP endpoint. Every sentence adds value, with no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description covers the main purpose and pagination parameters, it lacks details about the return structure beyond the preview_path field. Since no output schema is provided, the description should describe what fields to expect in each entry. This omission leaves the agent uncertain about the response format, making the context incomplete for a well-informed invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Both parameters (after, limit) are fully described in the input schema, achieving 100% schema description coverage. The description does not add any new meaning or usage guidance beyond what the schema already provides, so it meets the baseline without additional value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the action ('List pending project transfers'), the scope ('OFFERED TO the authenticated wallet'), and distinguishes from siblings like list_outgoing_transfers by emphasizing 'incoming'. It also adds version info and the API endpoint, leaving no ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use the tool (to view incoming transfers) but does not explicitly state when not to use it or suggest alternatives like list_outgoing_transfers or preview_project_transfer. The purpose is understandable, but usage guidance is implicit rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses key behavioral traits: authentication via SIWX wallet, error responses for org_id (403/400), pagination with limit/cursor, and the all parameter for cross-wallet inventory. However, it does not explicitly state that it is read-only or mention rate limits, but given no annotations, this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single paragraph that front-loads the main purpose and provides necessary details. It is relatively concise, though it could be slightly more structured (e.g., bullet points) without losing information. Every sentence contributes value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description lists returned fields (name, site_url, etc.), explains auth, error cases, and pagination. It covers the main usage scenarios and constraints, making it fairly complete for a listing tool with moderate complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds minimal meaning beyond the schema's parameter descriptions; it mostly repeats the same information (e.g., org_id authorize-before-reveal, all parameter). No significant extra context is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List projects' and specifies the scope (membership-scoped) and the endpoint. It distinguishes from siblings like project_get by focusing on listing multiple projects, with explicit details on returned fields.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool for listing projects, but does not explicitly mention when not to use it or provide alternatives such as project_get for single project retrieval. It gives context on filtering and pagination but lacks explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Mentions redaction and excluded data, but omits read-only nature, pagination behavior, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences efficiently convey purpose, endpoint, auth, and exclusions without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks output schema; description does not explain response structure, pagination usage, or record format, which is needed for a 4-parameter tool with no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and describes all parameters; description adds no additional meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'list redacted tenant x402 payment records for priced function web routes on a project' with HTTP endpoint, distinguishing it from other list tools by specifying the resource and redaction behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Specifies required permission scope ('project.tenant_payments.read: org developer+ or read-scoped project grant/delegate') and what is never returned, but lacks explicit guidance on when to use vs alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description mentions the requirement for reveal:true and that it emits secret material, which is critical. However, it does not describe side effects on the cache, return value format, or any security restrictions beyond the boolean flag.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two precise sentences, front-loading the purpose and adding necessary condition and use case. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (2 params, no output schema), the description covers the essential points but omits return value, error cases, and potential destructive behavior on the cache.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for both parameters, so baseline is 3. The description adds minimal context beyond the schema, only linking reveal=true to secret emission.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Export') and the resource ('local cached anon/service keys for one project'), and distinguishes from any sibling tools with similar purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies the prerequisite (reveal:true) and the context ('when a credential-required operation needs keys'), but does not explicitly mention when not to use or provide alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses auto-creation on first verification and rate limits (5/hr per email, per tier), which are important behavioral traits beyond what the schema shows.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, no wasted words. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 6 parameters (including enums) and no output schema, the description covers core behavior and important side effects but lacks details on return value, error conditions, or specific behaviors per intent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description does not add meaning to individual parameters like intent or delivery beyond the schema descriptions. Rate limits are mentioned but not tied to parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it sends a passwordless login email (magic link) to a project user and mentions auto-creation on first verification. This distinguishes it from sibling tools like verify_magic_link or create_auth_user.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for sending magic links and notes auto-creation, but does not explicitly guide when to use this vs alternatives (e.g., for explicit user creation vs. sign-in). With many sibling auth tools, more explicit guidance would be helpful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the prerequisite (active tier) but does not mention side effects, rate limits, or whether the operation is destructive. The behavior is minimally inferred.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words. The most critical information (action and prerequisite) is front-loaded. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with one parameter and no output schema. The description covers the action and a prerequisite but omits any mention of expected outcome, success signals, or error cases. It is adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the parameter description in the schema ('Message to send to the Run402 developers') matches the tool description almost verbatim. The description adds no additional meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'send' and the resource 'message to the Run402 developers'. This distinct purpose separates it from sibling 'send_email', which sends generic emails.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It mentions the prerequisite 'Requires an active tier', but does not explicitly provide when-to-use or when-not-to-use guidance or alternatives. The distinction from 'send_email' is implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behaviors: convergent (idempotent), tables dark by default, revocation of dropped items. No annotations provided, so description carries full burden and meets it well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with clear front-loading of action. Each sentence adds value, though slightly verbose. No redundant content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers manifest structure, convergence, and default behavior. Lacks explanation of return values, error conditions, or success/failure outcomes. Adequate for a mutation tool but could be more complete given complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. Description summarizes manifest structure but does not add new parameter syntax or format details beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'apply' and the resource 'declarative authorization manifest to a project', includes the HTTP endpoint. Distinguishes from sibling tools like 'validate_manifest' (validation) and 'get_expose' (retrieval) by focusing on application.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for applying manifests but does not explicitly mention alternatives or when not to use. The convergence note ('applying twice is a no-op') provides guidance but no direct comparison to sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral burden. It discloses key behaviors: wallet resolution via SIWX (signed automatically), and the admin key requirement for non-linked wallets. However, it does not mention rate limits, side effects, or whether it is purely read-only (though implied).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the primary purpose. Every word adds value, with no redundancy or filler. It efficiently conveys the core function and an important behavioral nuance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no output schema), the description covers the main concepts: purpose, wallet resolution, and access control. However, it omits the return format (e.g., what 'available and held funds' looks like), which would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides 100% coverage with a basic description. The tool description adds significant context: wallet resolution mechanism and admin key requirement, which go beyond the schema. This extra information helps understand parameter implications.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the action ('Check'), the resource ('organization balance for the agent's allowance wallet'), and the scope ('available and held funds'), making the purpose specific and clear. It implicitly distinguishes from sibling tools like 'allowance_status' by focusing on balance checking.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It mentions a condition (admin key for other wallets) but does not explicitly state use cases, prerequisites, or exclusions. No sibling differentiation is made.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so description carries full burden. It implies a read operation (get), but does not explicitly state safety, side effects, or authentication needs. Acceptable for a simple getter.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence efficiently conveys purpose and scope, with no wasted words. Information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description lists key return fields (subject, events, repo id, etc.), making it useful. Could mention error handling, but not required.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of the single parameter (binding_id). Description adds return field details but no additional parameter meaning. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get one CI/OIDC deploy binding by id' and lists specific fields returned, distinguishing it from sibling tools like ci_list_bindings and ci_create_binding.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when a single binding ID is known, but lacks explicit when-not or alternative conditions. Sibling tool names (ci_list_bindings) provide context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden. It mentions that route_scopes are included when delegated, which is a behavioral detail beyond a simple list. However, it doesn't disclose any side effects, authentication needs, or performance characteristics. It adequately describes a read operation without contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two sentences: the first states the action, and the second provides usage context. No unnecessary words. It is well-structured and front-loaded with the primary purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (one parameter, no output schema), the description is largely complete. It explains what is listed (including route_scopes when delegated) and when to use it. The lack of output schema is mitigated by the clear behavioral description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, project_id, is fully described in the schema with clear meaning. The description adds no additional information beyond what the schema provides. With 100% schema coverage, baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List', the resource 'CI/OIDC deploy bindings', and scope 'for a project, including route_scopes when delegated'. This distinguishes it from sibling tools like ci_create_binding, ci_get_binding, and ci_revoke_binding.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage context: 'Use this to inspect which GitHub Actions subjects can deploy before editing bindings.' While it doesn't explicitly state when not to use it, the context implies it's for inspection before editing, and sibling tool names provide differentiation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses ownership requirement and action type, but does not detail idempotency, side effects, or behavior on duplication. It is adequate but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise paragraph, front-loaded with the main action, and every sentence adds value. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (5 params, nested object), the description covers purpose, parameters, and access control. It lacks details on return values or error conditions, but is generally complete for a creation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description lists parameters with examples, but adds little beyond the schema descriptions beyond summarizing and providing an example for capability.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool issues a per-project capability grant to a wallet, targeting agent/CI principals that aren't broad org members. It uses a specific verb (Issue) and resource (grant), and distinguishes from the sibling revoke_project_grant tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies it is for agent/CI principals not in broad org membership and requires ownership of the project's org. However, it does not explicitly mention alternatives or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist. Description implies it's a read-only fetch but does not disclose auth requirements, error behavior, or rate limits. It is clear it returns event shapes similar to deploy.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with main action, no redundant information. Each sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, but description explains return type (DeployEvent shapes). Use cases are covered. Minor omission: no mention of ordering or pagination for the event stream.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are already documented. The description adds context about operation_id being from a prior deploy call but does not provide new semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'fetch' and resource 'recorded phase-event stream for a deploy operation'. It distinguishes from sibling tools like deploy and deploy_resume by specifying it's for after-the-fact inspection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description provides explicit use cases (inspecting a deploy after the fact, for observers or resumption) but does not explicitly state when not to use or compare to alternatives like deploy_list or deploy_release_get.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that when no release is active, it returns null and empties inventory, and preserves canonical SDK errors. With no annotations, the description carries full burden; it is decent but could explicitly state read-only behavior or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three efficiently front-loaded sentences with zero waste: purpose, null-case behavior, and usage context with error handling. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequately covers purpose, null behavior, use case, and error preservation for a simple 2-parameter tool without output schema. Minor gap: no description of the successful response structure beyond the null case.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the description does not add meaningful detail beyond the schema. The baseline of 3 is appropriate as the description adds no extra parameter insight.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Fetch the current-live release inventory for a project' with specific verb and resource. It distinguishes from siblings like deploy_release_diff by noting 'Use this before deploy diffs', making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises 'Use this before deploy diffs to understand what is currently serving', providing clear context. However, it does not explicitly state when not to use this tool or name alternative tools for other scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It explains files are staged, uploaded incrementally (only missing bytes), and served via CloudFront. It also notes pricing ('Free with active tier'). Could mention auth or rate limits but provides good behavioral context for a static deploy.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each adding value: what it does, how it works, and cost. No wasted words, major points upfront.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and 3 parameters, the description covers the deployment process, transport mechanism, and result (unique URL via CloudFront). Could mention project requirement but described in schema. Lacking error handling or edge cases, but adequate for a simple tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are already documented. The description adds that files are 'inline file bytes' but doesn't elaborate on parameter details beyond what the schema provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Deploy a static site (HTML/CSS/JS) from inline file bytes' with specific verb and resource. It details the process of staging, uploading via plan/commit transport, and serving via CloudFront, differentiating it from sibling tools like deploy_site_dir which likely use directory paths.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives such as deploy_site_dir or deploy_function. The description implies it is for static sites with inline files but lacks when-to-use or when-not-to-use advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description reveals that the tool switches addresses from managed fallback to custom domain, which is useful behavioral context. Without annotations, more information about idempotency, permissions, or side effects would improve transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the main action, no redundancy. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with two parameters and no output schema, the description adequately covers the action, precondition, and effect. It could mention idempotency or error cases for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter descriptions, so the description adds minimal extra meaning beyond mentioning the activation condition. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Activate' and the resource 'custom mailbox addresses', with a precondition 'once ProjectDomain receive checks have passed'. This distinguishes it from sibling tools like domains_check or domains_apply.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states the precondition for use ('once checks have passed'), providing clear context. However, it does not mention when not to use or directly compare to alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It discloses the creation behavior and the expected return of an address/token, but does not detail side effects, validity duration, or auth requirements beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences efficiently convey purpose and usage instructions with no wasted words. The description is well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given three parameters, no output schema, and no annotations, the description is fairly complete: it states the outcome (created token/address) and the workflow. However, it omission of error conditions or return format leaves minor gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all three parameters. The description adds context (mailbox address, test token) but does not enhance parameter semantics beyond what the schema provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates a test token for receive readiness testing, specifies the action (create), the resource (ProjectDomain receive test token for a mailbox address), and distinguishes it from sibling tools like domains_check by outlining the workflow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit follow-up actions: send mail to the returned address and poll domains_check for readiness. While it lacks explicit when-not-to-use or alternatives, the context is clear and guides usage effectively.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries burden. 'Inspect' suggests read-only, but no explicit confirmation of non-destructive behavior or authorization requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with key verb, no unnecessary words. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple get tool with one parameter and no output schema. Lists key return aspects, though response format is not described.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only parameter 'version_id' is documented in schema; description adds value by listing what the response includes (details, secrets, pricing), going beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'inspect' and resource 'specific published app', listing included details like secrets and pricing. Distinguishes from siblings like browse_apps.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies use for a single app via 'specific', but lacks explicit when-to-use vs alternatives like browse_apps or fork_app.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations present, so description must carry full burden. It discloses two return sources (applied vs introspected) and the source field, providing useful behavioral context for a read-only operation. Missing error handling or permission details but adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose and behavior. Every sentence adds value with no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with one parameter and no output schema, description explains both return paths and the source field. Could detail manifest structure but suffices.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter project_id. Description adds no additional meaning beyond the schema, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it retrieves the current authorization manifest for a project, distinguishing it from siblings like apply_expose (which applies) and validate_manifest (which validates). Includes endpoint and behavior details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for reading the current manifest but lacks explicit guidance on when to use this vs alternatives like apply_expose or validate_manifest. No 'when-not' or exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description discloses the non-revealing 403 error for non-members and that any active member can read. It lacks details on rate limits or idempotency, but the behavioral info is good for a simple read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, using a single sentence to convey the purpose, fields, access rules, and error behavior. Every part is necessary and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, read-only, no output schema), the description covers key aspects: what it returns, who can use it, and error handling. It could mention idempotency but is largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with one parameter. The description states 'Params: org_id' and the schema provides a description. No additional meaning is added beyond repeating the parameter name.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Read one organization', lists the specific fields returned, and includes the HTTP method and path. It distinguishes from sibling tools like list_orgs and create_org.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions that any active member may read and non-members get a 403, but does not explicitly state when to use this tool versus alternatives like list_orgs for multiple organizations or create_org for creation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the burden. It describes the output (tables, columns, etc.) and implies a read-only operation, but does not explicitly confirm non-destructive behavior, permissions, or performance characteristics. Adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first defines the operation, second gives usage context. No wasted words, front-loaded with key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite lacking an output schema, the description enumerates what is returned (tables, columns, etc.), which is sufficient for this tool. It integrates well with sibling tools and the simple parameter. Minor gap: no mention of output format or limits.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (one parameter fully described). The description adds no additional meaning beyond the schema; it does not elaborate on the parameter's use or provide examples. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'introspect' and clearly identifies the resource 'database schema' with enumerated details (tables, columns, types, constraints, RLS policies). It effectively distinguishes from query-execution siblings like 'run_sql' and 'rest_query'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states it is useful 'before writing queries', providing clear context for when to use. Does not specify when not to use or list alternatives, but the guidance is adequate for a straightforward introspection tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It describes what the tool reports but does not explicitly state that it is read-only or whether it modifies any files. The term 'inspect' implies safety, but more explicit behavioral disclosure (e.g., no side effects) would improve transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently packs all relevant information: target, reports produced, and key condition (no credentials). No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has only one parameter and no output schema, the description provides a good overview of what the tool does and what it returns. It could be more explicit about the return format (e.g., JSON) but is largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the description essentially repeats the schema's description of archive_path. No additional semantic detail is added beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Inspect' and the resource 'local archive directory or .r402ar tar offline'. It lists the reports produced (digest, secrets, etc.) and distinguishes from sibling tools like export_project_archive and verify_project_archive by emphasizing offline inspection without Cloud credentials.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'without Cloud credentials' and refers to local paths, indicating this tool is for offline inspection. It does not explicitly contrast with alternatives but the context from siblings and the clear scope imply when to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the full burden. It mentions platform management and idempotency but lacks details on auth needs, destructive potential, or process outcomes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two front-loaded sentences with zero waste. Every word serves a purpose, efficiently conveying the tool's function and key requirement.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and nested objects, the description adequately covers the request shape but omits expected return value or post-submission behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description reiterates required fields (job_type, input, max_cost_usd_micros) already in schema, adding little extra meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Submit a platform-managed job' with a specific verb and resource. It distinguishes from siblings like jobs_get and jobs_cancel by focusing on submission.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on the expected request shape but does not explicitly state when to use this tool versus alternatives or mention exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries full burden. It discloses a failure condition and the effect (enabling hybrid access), but does not mention authorization requirements, whether the operation is idempotent, or any other side effects. For a mutation tool, this is adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences covering purpose and a key behavioral note (failure condition). No redundant information, front-loaded with the main action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 required parameters, no output schema), the description covers the essential purpose and a notable failure case. It does not describe the success return value, but that is generally acceptable for a straightforward mutation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema provides full descriptions for both parameters (org_id and wallet), making the schema coverage 100%. The description adds context about the overall purpose but does not add new parameter-specific meanings beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (link wallet to organization), the resource (existing email organization), and the outcome (enabling hybrid Stripe + x402 access). It distinguishes from siblings by specifying the linkage operation, which no other sibling tool explicitly performs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description provides clear context that the organization must already exist (implied by 'existing email organization'), and mentions a failure condition (wallet already linked elsewhere). However, it does not explicitly state when NOT to use this tool or suggest alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry behavioral burden. It describes output fields but lacks details on pagination, ordering, or whether it includes all membership statuses. However, for a simple list tool, basic transparency is provided.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, clear and direct, no unnecessary words. Front-loaded with key purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple no-param tool, the description specifies returned fields. Missing details like authentication requirements or pagination, but overall adequate given simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist (empty schema, 100% coverage). The description adds no param info, which is acceptable. Baseline for 0 parameters is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states it lists orgs the user is a member of and specifies returned fields (id, display name, role, membership status). It clearly distinguishes from siblings like 'get_org' (single org) and 'list_org_members' (org members).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use when needing a list of user's orgs, but does not explicitly state when to use vs. alternatives or provide exclusions. No guidance on when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Given no annotations, the description carries the full burden. It discloses that new/changed emails trigger operator email reply verification and mentions 'Free with allowance auth', providing behavioral context about activation and authorization. However, it does not address whether the operation overwrites existing info or requires specific roles.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, with two sentences (or three clauses) that front-load the purpose and include only essential details. Every sentence contributes meaning, with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple setter with 3 params, no output schema, and no annotations, the description covers purpose, a key behavioral trait (email verification), and authorization/cost context. It lacks details on return value or error states, but is largely complete for the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with each parameter described in the schema. The description adds only a slight rephrasing of the email parameter's effect (new/changed emails start verification), which is already in the schema. Thus, minimal added value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Register' and the resource 'agent contact info', listing the fields (name, email, webhook). It distinguishes from sibling tools like get_agent_contact_status and verify_agent_contact_email, as it is the only setter.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for setting or updating contact info and mentions the side effect of email verification, but it does not explicitly state when to use this tool versus alternatives such as get_agent_contact_status (read) or verify_agent_contact_email (verification).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the mutation nature, required ownership, and a specific error case (409 LAST_OWNER). But it does not mention other potential side effects, idempotency, or whether changes take effect immediately. Without annotations, the burden is higher and the description is adequate but not fully transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, no wasted words, front-loaded with action and hierarchy, and efficiently covers key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple mutation tool with no output schema, the description covers the action, prerequisites, and a notable error case. It lacks return value information but is otherwise fairly complete given the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already describes all parameters. The description adds context for the role hierarchy and notes that principal_id comes from list_org_members, but the schema's role enum description already includes hierarchy. Thus the description adds minimal value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Change a member's role' and includes the role hierarchy (owner > admin > developer > billing > viewer), which distinguishes it from sibling tools like add_org_member, remove_org_member, promote_user, and demote_user.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It specifies the precondition 'Requires an active `owner` membership' and a failure condition for demoting the only owner. However, it does not explicitly compare to similar tools like promote_user or demote_user, which could help agents choose the right tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description compensates by disclosing that prototype projects are locked to 'run402_transparency' and attempting 'none' surfaces a specific typed error, revealing important behavioral constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with three sentences, no excess, and the main purpose is front-loaded. However, structure could be improved with bullet points for constraints.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 3 parameters and no output schema, the description covers input constraints well but does not mention the return format or success behavior, leaving some context missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds meaning by explaining the supported footer_policy values and the prototype lock, which are not fully covered in the schema's enum description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates per-mailbox settings, specifically the footer_policy, and distinguishes itself from other mailbox-related tools (create, delete, get) by focusing on updates.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus alternatives, but the name and context imply it's for modifying mailbox settings. No exclusion criteria or alternative tool references are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses that enabling sets lease_perpetual: true, prevents lifecycle advancement beyond active, projects inherit the state, enabling on grace-state organizations reactivates inline and returns reactivated: true, and that it uses the configured allowance wallet. It omits details like idempotency or error responses but covers essential behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is moderately concise with no fluff. The first sentence clearly states the main action. It efficiently conveys the behavior, auth requirements, versioning, and API endpoint. Could be slightly more structured (e.g., bullet points) but remains effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity, the description is fairly complete. It explains the core functionality, project inheritance, reactivation behavior, admin authentication, and the replacement of the old endpoint. With no output schema, it mentions the reactivated: true field in responses. It lacks information about potential errors or pagination but covers the necessary context for an agent to use it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both parameters well-described. The tool description adds conceptual context ('escape hatch', 'pinned state', 'v1.57+') but does not significantly extend beyond the schema descriptions for the parameters themselves. The schema already explains format, admin requirements, and effects.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action: 'Toggle an organization's `lease_perpetual` escape hatch'. It specifies the resource (organization), the effect (pinning state, inheritance, reactivation), and distinguishes itself from the deprecated v1.56 pin_project. Among siblings like admin_archive_project and admin_reactivate_project, this tool's unique purpose is well-defined.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use the tool: for toggling the lease_perpetual flag on an organization. It specifies that it is platform-admin only using an allowance wallet, and that it replaces the older pin_project endpoint. However, it does not explicitly state when not to use it or list alternatives for similar admin tasks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses the tool's behavior: it creates a new allowance, generates a private key, derives an address, and writes to a specific file. This is transparent for a simple creation action, though it could mention any implicit permissions or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two well-structured sentences. It front-loads the primary purpose and then provides key details. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, no output schema), the description covers all necessary information: what it does, what it creates, and where it saves. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and the input schema coverage is trivially 100%. The description adds no parameter information, which is acceptable per the rubric baseline of 4 for no parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a new local agent allowance on Base Sepolia testnet. It specifies the actions: generating a private key, deriving an Ethereum address, and saving to a config file. This distinguishes it from sibling tools like allowance_export and allowance_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention conditions, prerequisites, or when not to use it. Given the presence of related sibling tools, explicit usage context would be beneficial.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It clearly states the tool is read-only, requires no signing/gas/billing, and is a 'pure RPC convenience'. Missing details on failure modes or behavior if a non-view/pure function is provided, but overall adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences that front-load the key purpose ('Read-only smart-contract call') and pack essential behavioral traits. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description does not mention what the call returns. For a read-only tool, understanding the output format is important. While the schema coverage is high, the lack of output schema information leaves a gap for agents to infer the response structure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for all 5 parameters, so the bar is low. The description adds no additional parameter-specific details beyond noting the function is view/pure, which is already implied by the tool's purpose. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it is for read-only smart-contract calls (view/pure functions), explicitly distinguishing it from write operations. The sibling tools include 'contract_call' and 'contract_deploy', which are for write or deploy actions, making the purpose distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates when to use this tool: for read-only calls with no signing, gas, or billing. It implies that for other operations, sibling tools should be used, though it does not explicitly state alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It states the output is written to a local file and that parent directories are created. However, it does not disclose if the file is overwritten, the potential for long downloads, or any side effects beyond writing. Some gaps remain.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two tightly-written sentences. The first sentence immediately defines the primary purpose, and the second provides necessary context about prerequisite data and legacy information. No filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains the tool's purpose and parameter semantics well, but it does not cover the return value (if any) or error scenarios (e.g., what happens if the file doesn't exist). Given no output schema and no annotations, some completeness is missing, though the core usage is clear.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, providing a baseline of 3. The description adds value beyond the schema by specifying that job_id must be from a completed job, that filenames come from jobs_get's artifacts map, and that output_path will create parent directories. This extra context improves parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('download') and the specific resource ('completed managed job's artifact by filename to a local file'). It distinguishes this tool from siblings by mentioning that filenames are discovered via jobs_get, and it notes that legacy refs are retired, providing clear differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit context for when to use the tool: after a managed job is completed and after retrieving filenames from jobs_get. It implies the job must be completed (reinforced in parameter description). It does not explicitly list when not to use it, but the guidance is sufficient for typical use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description fully carries the burden. It discloses what the return payload contains (including sensitive data like secret names but not values) and the authentication requirement. It does not mention rate limits or side effects, but for a read-only preview this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, but the first sentence is long and lists many return fields. It is still readable and front-loaded with the core action. Could be slightly more concise, but not wasteful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the single parameter, no output schema, and no annotations, the description provides a thorough list of return fields, version requirement, and caller prerequisite. It is nearly complete for guiding an agent, though error conditions could be added.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter transfer_id already has a detailed description in the input schema (100% coverage). The main description adds nothing beyond the schema, so it meets the baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Fetch' and the resource 'preview document for a project transfer', distinguishing it from sibling transfer tools like accept_project_transfer or initiate_project_transfer. It also mentions the API endpoint for precise identification.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies that the caller must be a party to the transfer and that the tool is kind-agnostic (works for wallet and email transfers). Although it does not explicitly contrast with siblings, the context makes it clear this is a preview step before acting.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that the tool is read-only, returns redacted data, is local-only, and never returns full keys. Missing permission or latency details, but adequate for a simple read tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no fluff, front-loaded with purpose. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Fully describes what the tool does, its limitations, and what it returns, despite no output schema. Sufficient for correct use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema already describes project_id fully (100% coverage). Description adds no new parameter-specific info beyond context about cache nature.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'Read' and specific resource 'redacted local project-key cache status', distinguishing it from siblings like project_key_cache_export and project_get.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context that it's local cache and not authoritative, but does not explicitly compare to alternative tools. Implies usage for cache status only.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears the burden of behavioral transparency. It clearly states the effect (set or clear address, used for auto-drain on day-90 deletion) and that the address is optional. It does not contradict any annotations. Some additional detail on side effects could further raise the score.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single concise sentence that conveys the essential action and context without unnecessary words or repetition. Well-structured and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with only three required parameters and no output schema, the description provides sufficient context about the purpose and use case. Nothing essential is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All three parameters have full descriptions in the input schema (100% coverage), so the description adds no new semantic information beyond restating the schema. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (set or clear) and the resource (recovery address for a KMS signer) and provides the specific context (auto-drain on day-90 deletion). It distinguishes this tool from siblings like drain_signer.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when managing recovery addresses but lacks explicit guidance on when to use versus alternatives like drain_signer. No prerequisites or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses key behaviors: creates user if nonexistent, token is single-use, expires in 15 minutes. It also indicates return values (access_token, refresh_token). This covers side effects and constraints well, though it could mention rate limits or authorization requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each serving a purpose: main action, side effect, constraints. No wasted words. Information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description specifies return values (access_token, refresh_token). It covers behavior, side effects, and constraints. For a tool of this complexity, the description is fully informative.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description does not add new meaning beyond the schema's parameter descriptions. It merely restates 'code' and 'token' in the context of exchange, which is already clear from the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verbs and resources: 'Exchange a magic link token for access_token + refresh_token'. It also adds unique behaviors (user creation, single-use, expiry) that distinguish it from nearby tools like 'request_magic_link' which likely sends the link, and other auth tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (when you have a magic link token) but does not explicitly state alternatives or when not to use. For example, it doesn't mention that this is for verifying the link after the user clicks it, nor does it contrast with other authentication methods like password or passkey.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Given no annotations, the description carries the burden. It discloses that a brand-new wallet is provisioned as a 'human' principal and the role defaults to 'developer'. It does not mention reversibility, errors, or rate limits, but the side effect is clearly stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single paragraph of four sentences, each serving a purpose: action, side effect, default, prerequisite, and alternative. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no output schema, the description covers the action, prerequisite, default, and side effect. It could mention what the response indicates (e.g., success or the new member object), but this is a minor gap. The presence of list_org_members among siblings compensates partially.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds meaningful context: that 'wallet' can be an EVM address or named wallet, that a new wallet is provisioned, and that 'role' defaults to 'developer'. This exceeds mere schema repetition.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The verb 'Add' with resource 'member to an org' is explicit. The method 'BY WALLET' and inclusion of the REST endpoint clarify the operation. It distinguishes from the email-first invite flow, differentiating from potential sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly requires an active 'owner' membership and notes the default role. It mentions the email-invite alternative as separate and not-yet-shipped, providing context for when to use this tool. However, it does not explicitly name alternative tools or state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries the full burden. It discloses that bytes are written directly to disk and that return includes size and SHA-256 header (if stored). This adds value beyond the input schema, though it does not mention file overwrite behavior or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states action, second explains return. Every word adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple download tool with 3 parameters and no output schema, the description covers purpose, behavior, and return values. However, it lacks information on error handling (e.g., missing key) and overwrite behavior, leaving minor gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides descriptions for all three parameters (100% coverage). The description adds no further detail about parameters, so it meets the baseline but does not exceed it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description explicitly states 'Download a blob to a local file path', with a specific verb and resource. It differentiates from sibling tools like assets_ls, assets_put, assets_rm, assets_sign by focusing on download.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description provides clear context by mentioning 'no context-window bloat' as a benefit, implying use when local storage is preferred. However, it does not explicitly state when not to use or compare to alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses the time-bound nature, default TTL (1 hour), maximum (7 days), and that it is a presigned GET URL (read-only). It doesn't specify error behavior if blob is missing, but the key behavioral traits are covered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: the first states the core functionality, the second provides usage guidance and default/max values. No wasted words, front-loaded with essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple 3-parameter tool with no output schema, the description is largely complete. It explains purpose, usage, and key parameter constraints. It could mention that the URL is returned, but that is implied by 'generate... URL'.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter descriptions. The description adds context about default TTL and max TTL, but this is already partially covered in the schema. Baseline 3 is appropriate as the description adds marginal value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool generates a time-boxed S3 presigned GET URL for a blob, with a specific use case (external sharing without exposing API key). It distinguishes itself from sibling assets tools like assets_get, assets_ls, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says to use this for sharing private blobs externally without exposing the apikey. It provides clear context but does not explicitly exclude cases where alternative tools (e.g., assets_get) might be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility. It discloses authorization, error handling, and the HTTP endpoint. It does not detail side effects or reversibility, but covers key behavioral aspects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences, front-loaded with the primary action. No unnecessary words, each sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and related sibling tools, the description covers purpose, auth, error conditions, and endpoint. It does not explain post-cancellation behavior or notification, but is adequate for the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description does not add meaning beyond the schema, as the parameter descriptions already exist and are not enriched further.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Cancel') and the resource ('pending project transfer'), including a version requirement. It distinguishes from siblings like 'accept_project_transfer' and 'claim_project_transfer' by specifying cancellation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies authorization requirements and error conditions (409 for already-processed transfers). It lacks explicit comparison to siblings but provides sufficient context for when to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description covers output and error behavior but doesn't disclose side effects, auth needs, or rate limits. Basic transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, no fluff. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequately describes return value and error behavior for a tool with no output schema. Could elaborate on diff bucket structure but sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all 4 parameters with descriptions. Description adds extra meaning by specifying valid values for 'from' and 'to' and mentioning 'limit' default, going beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'diff', resource 'release targets', and scope 'for a project'. Specifies valid values for parameters, distinguishing it from sibling tools like deploy_release_active.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context on parameter values and mentions error preservation. Lacks explicit alternatives or when-not-to-use, but sufficient for typical use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses key behaviors: idempotency, that only the failed phase is re-run, that SQL is never replayed, and the outcome when called on an already-terminal operation. This is good but could mention return value structure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loading the key condition and purpose. Every sentence contributes meaningful information without any fluff or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the single parameter and no output schema, the description covers the essential context: states to resume from, idempotency, and a typical failure scenario. It lacks details on possible errors or return format, but these are not critical for agent invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a single parameter (operation_id) described as 'Operation id returned by a prior `deploy` call. Required.' The description adds no new information about the parameter beyond what the schema provides, justifying the baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool resumes a deploy operation that ended in specific states ('activation_pending' or 'schema_settling'), using specific verbs and resource. It distinguishes from siblings by defining its precise role in the deploy lifecycle, even without explicit comparison.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear conditions for use (resuming a deploy in certain states) and includes a concrete example (transient gateway failure). However, it does not explicitly contrast with other deploy tools or state when not to use it, though the context implies its specific utility.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses that the tool is safe, provider-managed, and does not mutate external DNS for manual-DNS domains. This provides adequate behavioral traits for decision-making.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences, front-loaded with the primary action. Every sentence adds distinct value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (different behaviors for provider-managed vs. manual-DNS) and no output schema or annotations, the description covers the main use cases and key exception. It is mostly complete, though could mention response or prerequisites.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for both parameters (domain, project_id). The description adds no further parameter details beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool applies safe provider-managed ProjectDomain changes, and distinguishes from manual-DNS handling. Among sibling tools like domains_activate, domains_check, etc., this description uniquely identifies the purpose and scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It specifies when to use (when authority is available) and describes a different behavior for manual-DNS domains, implicitly guiding the agent to use alternative tools for manual-DNS. However, it does not explicitly name alternative sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully carries the burden of behavioral disclosure. It details what data is returned (desired, observed, effective, DNS records, checks, next actions), indicating a read operation. It does not mention potential side effects, but the description is sufficiently transparent for a read tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no redundant words. It is front-loaded with the action and then details the returned data. Every word is necessary and contributes to understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 required params, no output schema), the description adequately conveys the purpose and return data. It could mention error cases or empty results, but overall it provides sufficient context for an AI agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for both required parameters ('domain' and 'project_id'). The description does not add additional semantic meaning beyond what the schema already provides, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and the resource ('one ProjectDomain aggregate') with specific included fields (desired, observed, effective, DNS records, checks, next actions). It distinguishes this tool from sibling domain tools like 'domains_list' (which likely returns multiple) and action-oriented tools like 'domains_activate'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies that this tool should be used when you need detailed aggregate information for a single domain, contrasting with 'domains_list' for multiple. However, it does not explicitly state when not to use it or mention alternatives for other domain operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden and adequately discloses that the tool repairs infrastructure without modifying external DNS. It does not detail potential side effects or permissions, but the main behavioral scope is transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, front-loaded sentence that efficiently conveys action, scope, constraints, and example without any filler. Every part earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the input schema (two parameters) and absence of output schema, the description provides sufficient context about purpose and constraints. Could benefit from mentioning return type or error handling, but overall complete for this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters (domain and project_id). The description adds context about the repair scenario but does not enhance parameter meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (repair), specific context (Run402-owned ProjectDomain infrastructure), provides an example (missing SES receipt-rule recipients), and a key constraint (without changing external DNS), effectively distinguishing it from sibling tools like domains_check or domains_apply.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (for repair scenarios) and what it does not do (no external DNS changes), but lacks explicit guidance on when not to use or alternatives among the many sibling tools, though the context is sufficiently clear for an agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full burden. It discloses the cost and that the output is a PNG image. The prompt length limit is inherited from the schema. It does not mention error handling or idempotency, but for a generative tool this is reasonable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states purpose, second adds cost and aspect ratios. No extraneous information. Front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description does not explain the return format (e.g., URL or base64). It covers key usage constraints (cost, aspect ratios) and input limits. Missing response details, but acceptable for a simple generation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%. The description repeats the aspect ratio enum and the prompt length, adding no new semantic information beyond the schema fields. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Generate a PNG image from a text prompt,' which specifies the verb, resource, and input. It is distinct from sibling tools (no other image generation tools). Includes additional details like cost and aspect ratios.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions a cost of $0.03 USDC via x402, which is a key usage constraint. It also lists available aspect ratios. However, it does not explicitly state when not to use or mention alternatives, but no alternatives exist in the sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description only states it reads preferences. It does not disclose any behavioral traits like authentication requirements or side effects, but the read nature is transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with the verb and resource, and efficiently lists all relevant preference aspects. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description covers the high-level content of preferences. It does not specify return format but is sufficient for a simple read operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (no parameters). The description adds meaning by listing the preference categories, providing context beyond the empty schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it reads the operator's notification preferences and lists specific categories (channels, cadence, toggles, locale, timezone), distinguishing it from sibling tools like set_notification_preferences and list_notifications.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for viewing current preferences, but does not explicitly state when to use versus alternatives or provide exclusions. However, the context is clear enough for differentiation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. The verb 'list' implies a read-only operation, and the description lists the returned fields without signalling any side effects. It is transparent about what the tool does, though it could explicitly state that it does not modify data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the purpose and includes a usage hint. It is concise with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple listing tool with one parameter and no output schema, the description is fairly complete. It explains what fields are returned and gives a usage context. It could mention pagination or ordering, but given the simplicity, it is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter project_id, with a clear description in the schema. The tool description does not add any additional information about the parameter beyond the schema, so baseline score 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'List' and resource 'mailboxes', and enumerates the fields returned, such as address/managed_address, readiness, and footer-policy fields. It clearly distinguishes from siblings like get_mailbox (singular) and list_projects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states 'Use before choosing or repairing email defaults,' providing a clear when-to-use hint. However, it does not explicitly mention when NOT to use it or alternative tools like get_mailbox for a single mailbox.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds significant detail beyond annotations: payment challenge auto-satisfaction, default payment ceiling, allowance wallet usage, idempotency key forwarding, and response contents (HTTP response + receipt).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single focused paragraph, front-loaded with core action, no redundant phrasing. Every sentence adds relevant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, payment mechanism, idempotency, return value. Lacks detail on error scenarios or behavior when URL does not support x402. Otherwise complete for a tool with no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all parameters with descriptions (100% coverage), so baseline applies. Description reinforces defaults (e.g., $0.10 ceiling) and idempotency behavior but does not add new meaning beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it calls an HTTP URL with automatic payment challenge satisfaction, distinguishing it from siblings focused on other operations. It specifies the action, resource, and unique capability.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context like default ceiling, allowance wallet, and idempotency key forwarding but lacks explicit guidance on when to use vs alternatives or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It clearly indicates this is a read operation ('read') and a live API call, implying no destructive side effects. It doesn't detail error handling or rate limits, but for a simple read tool this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long. The first sentence introduces purpose and key data points, the second clarifies the live nature and key exclusion. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple tool (1 param, no output schema), the description covers what it does, what it returns, and differentiation from siblings. It lacks details on return format or errors, but overall sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already covers 100% of parameters (project_id with description). The description adds minimal extra meaning beyond restating that it's an authoritative server view and no keys. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it's an 'Authoritative server read of a project' and lists specific data returned (name, org, tier, status, deploy, mailbox, usage). It distinguishes from sibling tools by explicitly noting it returns no keys and directing to project_key_cache_status/export for cached keys.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context ('Live API call') and directs users to alternative tools for cached keys. While it doesn't explicitly state when not to use this tool, the context is clear enough for an agent to decide.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description discloses key behaviors: KMS-backed, non-custodial, cost structure, and credit requirement. It does not mention synchronization or completion time, but covers the most important aspects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with four sentences, each adding distinct value: purpose, security, cost, requirement. Front-loaded with the main action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (provisioning with costs and security), the description covers essential points. It lacks details about return value or post-creation state, but is otherwise complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are already documented. The description adds context about costs but does not significantly enhance parameter understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Provision' and the resource 'AWS KMS-backed Ethereum signer' with its purpose 'for signing smart-contract write transactions'. This distinguishes it from sibling tools like get_signer, delete_signer, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context including cost and prerequisite ($1.20 cash credit). However, it does not explicitly state when not to use it or mention alternatives like existing signers.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description discloses important behavioral traits: required owner membership and failure on removing the only active owner. It could add more details on side effects or idempotency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the action, and every sentence adds value without unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple removal tool with 2 parameters and no output schema, the description covers core behavior, prerequisites, and an error scenario. Lacks detail on return value or atomicity, but sufficient for most contexts.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. The description adds no new parameter info beyond the schema; the error mention is already in the principal_id description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('remove a member from an org') and the resource (org member), distinguishing it from siblings like add_org_member and set_org_member_role.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context: prerequisite (active owner membership) and a specific failure condition (LAST_OWNER). However, it does not explicitly state when not to use it or compare to alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the rate limit, the amount of USDC returned, and its purpose (enough for 2 prototype databases). It does not mention error handling or consequences of exceeding rate limits, but is adequate for a simple faucet.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, each serving a purpose: stating the action, providing the rate limit, and specifying the return value. There is no redundancy or irrelevant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one optional parameter, no output schema), the description covers the essential aspects: what it does, constraints, and expected result. It does not detail the response format, but that is less critical for a prototype tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage for the single optional parameter 'address'. The description does not add additional meaning beyond what the schema already provides ('Wallet address... If omitted, reads from local agent allowance file.'). Following the baseline rule, score is 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('request'), the resource ('free testnet USDC from the Run402 faucet on Base Sepolia'), and the amount returned. It uniquely identifies the tool among siblings, as no other tool serves a similar faucet function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit rate limiting (1 per IP per 24h) and the return amount, which guides usage. Although it does not mention alternatives or when not to use, the tool's single-purpose nature makes usage obvious.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses auth requirement and return fields, but does not explicitly state it's read-only or mention any side effects, rate limits, or further behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with purpose, no wasted words. Efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple status check with no params and no output schema, the description adequately covers purpose, return fields, and auth requirement. Could mention if there are any limits or pagination, but not necessary for this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Zero parameters in schema, so baseline is 4. Description adds no parameter info, which is acceptable as there are none.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (Check) and resource (current tier subscription), listing specific returned fields (tier name, status, expiry, usage, caps). Distinguishes from siblings like allowance_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a clear prerequisite: 'Requires allowance auth.' Implies usage for checking subscription info, but does not explicitly compare to related tools like allowance_status.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description clearly states the tool only checks integrity and compatibility and treats archives as untrusted input, implying no side effects. This is sufficient for a verification tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no unnecessary words. Front-loaded with the core action. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the single parameter, no output schema, and no annotations, the description covers the tool's purpose, input type, and a key behavioral trait (untrusted input). Could mention return type or error handling but overall sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter archive_path. The description adds value by clarifying acceptable types (local directory or .r402ar tar path), which is not in the schema description beyond 'path.'
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it verifies a local archive directory or .r402ar tar file, checking integrity and compatibility. It distinguishes from siblings like inspect_project_archive by specifying it only verifies and does not trust input.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for offline verification but does not explicitly guide when to use this tool over alternatives like inspect_project_archive, import_project_archive, or export_project_archive. No when-not-to-use conditions given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses behavioral traits: stops future requests only, does not affect past releases or secrets. No annotations provided, so description carries full burden; it does so adequately.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states action, second clarifies scope and limitations. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple revocation tool with one parameter and no output schema, description covers purpose, effect, and limitations completely.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description for binding_id already provides meaning (CI binding id to revoke). Tool description repeats this without adding new details. Baseline 3 for 100% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action (revoke) and resource (CI/OIDC deploy binding). Distinct from siblings like ci_create_binding, ci_get_binding, ci_list_bindings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Describes effect (stops future requests) and limitations (does not undo releases or rotate secrets). Implicitly tells when to use; could explicitly state when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden and provides good behavioral context: it explains rule matching logic (AND, wildcards, one binding per rule) and implies a read-only operation. Lacks details on permissions or rate limits, but sufficient for a list tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences only: first states the purpose, second explains rule structure. No superfluous words, clearly front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains rule semantics but does not describe the output format (e.g., array fields, pagination). With no output schema, more detail would benefit completeness. However, the essential behavior is conveyed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters and 100% coverage. The description does not need to add parameter information; baseline is 4 for zero-parameter tools. No contradiction or omission.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the resource ('operator's Telegram routing rules') and the action ('List'). It explains the rule semantics (AND-ing dimensions, wildcard omitted dimensions, one binding per rule), which distinguishes it from siblings like list_notifications or list_notification_channels.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly conveys usage for viewing routing rules but does not explicitly state when to use this tool versus create/delete/modify alternatives. However, the context of sibling tools makes the use case clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries burden. It discloses auth requirements and response fields, but could mention reversal or further side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise, front-loaded with main action, no redundant sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple update tool with two required params, the description includes auth context, clearing behavior, and return fields, making it complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so description adds minimal extra meaning beyond what's already in the parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Set or clear an organization's display label' with HTTP method and endpoint, distinguishing it from sibling tools like rename_project.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context: 'Owner-only + step-up gated' and how to clear the label, but doesn't explicitly mention when not to use or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Covers auth mechanism, authorize-before-reveal with 403, validation rules, and SIWX auth requirement. No annotations exist, so description carries full burden and does well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three well-structured sentences with no fluff. First sentence gives purpose, second auth details, third security and validation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Missing return value description. For a rename tool without output schema, success/failure behavior beyond auth should be explained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with good descriptions. Description adds validation details but doesn't significantly enhance beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Rename a project' with HTTP method and resource. Specifies use case 'fix an auto-generated name', distinguishing it from other project tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides authorization context (org-membership, grant, SIWX auth) and security behavior. Does not explicitly list alternatives but context implies when to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description discloses key behavioral traits: two modes, default mailbox fallback, and result echoing mailbox_id and from_address. It does not mention error handling, rate limits, or idempotency, but it covers the main behaviors adequately.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only 4 sentences, front-loaded with the core purpose, and free of fluff. Each sentence adds critical information (modes, constraints, mailbox behavior, result). Perfectly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 11 parameters and no output schema, the description covers the main modes and mailbox behavior. However, it fails to mention that attachments are only allowed in raw HTML mode (described in schema but absent here). This is a notable gap for a tool with no annotations and high schema coverage, leaving the agent to infer a key constraint.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds value by explaining the two-mode concept, which clarifies parameter grouping (e.g., template and variables vs subject+html+attachments). It also reiterates mailbox defaulting, but the schema already does that. Overall, it provides a useful framework for parameter selection.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool sends an email, specifies two modes (template or raw HTML), and includes key constraints like single recipient and mailbox defaulting. It distinguishes itself from sibling tools like 'send_message' by explicitly naming the action and email mode options.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description guides usage by detailing two modes and their respective parameters (template vs subject+html), and it mentions single-recipient only. However, it does not explicitly compare to alternatives like 'send_message' or state when not to use it, leaving some ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It reveals the mutation (archived_at to NULL), a behavior change in v1.57, and the API endpoint. However, it does not discuss idempotency or rate limits, but for a simple field flip, this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with primary action. Each sentence adds value: action, version constraint, alternatives. No fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool with no output schema, the description covers authorization, limitations, and alternatives. Lacks example output or error cases, but overall complete for its complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds little beyond the schema; both state the same information about the project ID. No extra semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool un-archives a project by flipping `projects.archived_at` to NULL. It distinguishes from sibling tools like `admin_set_lease_perpetual` and `tier_set` by noting it does not handle organization-level lifecycle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states 'Platform-admin only' and provides clear alternatives: for grace-state organizations, use `tier_set` or `admin_set_lease_perpetual` instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses key behavioral traits beyond what annotations (which are absent) would provide. It explicitly states that the tool does not sign or broaden authority, and clarifies that route_scopes are required for route deployment. It could be improved by mentioning what happens on success or error, but overall it is transparent about the tool's limitations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (3 sentences) and front-loaded with the purpose. Every sentence adds essential information without redundancy. The structure is logical: purpose, behavioral note, and a specific caveat about route_scopes.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the high parameter count (10) and no output schema or annotations, the description covers the core behavioral aspects well. It explains the delegation flow and parameter importance. However, it lacks any indication of the return value or error conditions, which would be helpful for an AI agent to understand the tool's outcome.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3. The description adds value by explaining the role of the signed delegation (locally signed, not signed by the tool) and emphasizing the significance of route_scopes for route deployment. This additional context helps an AI agent understand parameter nuances beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: creating a GitHub Actions CI/OIDC deploy binding. It uses a specific verb ('Create') and names the resource ('GitHub Actions CI/OIDC deploy binding'). It distinguishes itself from sibling CI tools (get, list, revoke) by specifying it is a creation operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use the tool (to create a binding) and includes important usage notes: the signed delegation must be created locally, and without route_scopes the CI cannot deploy route declarations. However, it does not explicitly list alternatives or situations where this tool should not be used.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses important behavioral traits: the wildcard behavior of omitted parameters, the AND logic for matching, and the 404 error for invalid binding IDs. However, it does not mention idempotency or potential side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core purpose, and every part adds value. No unnecessary repetition or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description adequately covers input behavior, matching logic, error cases, and prerequisites. It is complete for a create tool with 5 parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds significant meaning beyond the schema by explaining the AND logic for matching criteria and the wildcard behavior (omitted = wildcard, empty array matches nothing). It also mentions the passkey requirement, which is not in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: creating a Telegram routing rule with matching criteria and a target binding. It distinguishes itself from sibling tools like list_notification_rules and delete_notification_rule by specifying creation behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions a prerequisite (operator_passkey assurance) but does not provide guidance on when to use this tool versus alternatives like list_notification_rules or test_notification. Usage context is implied but not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries full burden. It discloses the 7-day minimum window for AWS KMS deletion, the balance condition, and the refusal behavior. However, it does not mention permissions required, whether deletion can be reversed within the window, or the return value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is front-loaded with the core action and includes key constraints. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive tool with no output schema, the description covers the main behavioral aspects: action, time window, and precondition. It could be improved by mentioning whether the deletion happens automatically after 7 days, but it is sufficiently complete for an AI agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%. The signer_id description in the schema repeats the tool description, adding little new value. The description does not provide additional per-parameter details beyond what is in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool schedules the KMS key for a signer for deletion with a 7-day window, and mentions a precondition about balance. It matches the name 'delete_signer' and distinguishes from sibling 'drain_signer'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Refused if the signer has on-chain balance ≥ dust — drain first', which guides the agent to use the sibling 'drain_signer' tool before this one. It provides clear context for when to use and when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: the probe is from a single region, the probe may warm the cache (probeMayHaveWarmedCache), and errors for unauthorized or invalid URLs. This provides strong transparency about side effects and limitations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured paragraph that front-loads the core function, then adds usage, output details, and caveats. Every sentence contributes value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers input parameters, output fields (even without output schema), error cases, and behavioral notes. It is comprehensive for a diagnostic tool, though it does not compare with sibling tools like 'deploy_diagnose_url'.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for both parameters (project_id, url). The description adds context about the probe location and output fields, but does not enhance parameter semantics beyond what schema already provides, warranting a baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns live CDN state for a public blob URL, specifies it's from a single region (gateway-us-east-1), and contrasts with a global view. It also mentions specific output fields, making the purpose very distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use this tool (wrong version, suspected cache staleness) and provides error conditions (403 for unauthorized projects, 400 for invalid domains). It does not explicitly reference sibling tools, but the guidance is specific and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description holds full burden. It discloses that bytes are base64-encoded, bit-identical to DKIM original, no parsing/normalization/CRLF cleanup, inbound-only, and outbound returns 404. However, it does not specify the response format (e.g., JSON wrapper) or other error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, no fluff. First sentence states function, second adds key behavioral detail, third provides usage guidance and alternative. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description covers purpose, usage, and key behavioral constraints. Missing details about response structure (e.g., JSON wrapping) and error handling beyond outbound 404, but overall adequate for a simple retrieval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for all three parameters. The description repeats schema info (e.g., 'must be an inbound message') without adding new meaning. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Get the raw RFC-822 bytes of an inbound email message, base64-encoded' which clearly identifies the verb (get) and resource (raw bytes of inbound email). It distinguishes from sibling tool 'get_email' by specifying that this is for raw bytes vs. display/threading.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states use cases: 'Use this for cryptographic verification (DKIM checks, zk-email proofs)' and when not to use: 'Inbound messages only; outbound returns 404' and 'For display/threading, use get_email instead'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds value beyond annotations by explaining free vs paid behavior, idempotency_key semantics, error handling with isError=true, and the polling mechanism. Annotations are generic (readOnlyHint=false, idempotentHint=false), so the description provides necessary behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (3 sentences), front-loaded with the core purpose, and every sentence adds necessary detail. No redundant or irrelevant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (9 parameters, no output schema), the description covers key behaviors: free vs paid, idempotency, error handling, and polling. It lacks a description of the return value structure beyond 'result or pollable run_id', but it is sufficiently complete for most use cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are well-documented. The description adds meaning to idempotency_key ('reuse for same paid intent') and wait parameter (polling with idempotency_key replay), which goes beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Invoke' and resource 'deployed function via HTTP', distinguishing it from sibling tools like deploy_function or delete_function. It is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use this tool: it distinguishes free vs paid functions, explains idempotency_key requirement, and mentions polling behavior. However, it does not explicitly contrast with alternatives or state when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses critical behavior: single retrieval, 24h overlap, authentication requirement. With no annotations, it covers essential safety and mutation aspects well, though could elaborate on effect of repeated calls.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. Front-loaded with action, then key constraints. Every sentence provides unique value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequately covers behavior and prerequisites for a zero-parameter tool with no output schema. Minor gap: does not describe output format or type of the returned secret.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, baseline is 4. Description adds context about the operation (single return, grace period) that complements the empty schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Generate a fresh HMAC signing secret') and the specific resource ('operator's webhook endpoint'). It differentiates from sibling webhook tools by specifying 'operator's' rather than mailbox webhooks.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides key usage constraints: secret returned exactly once, previous valid for 24h, requires operator_passkey. Implicitly guides when to use (for rotation) but does not explicitly contrast with alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses the three behavioral modes and the project-level 'allow_password_set' requirement, though lacks details on side effects or authorization specifics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences front-loaded with the main purpose; no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema or annotations, the description covers the main behavioral modes and parameter relationships well, though it omits success/failure indicators.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (baseline 3). The description adds meaning by explaining which parameters to use in each mode (e.g., current_password for change, omit for reset/set).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states three distinct operations (change, reset, set) with specific conditions, clearly distinguishing the tool's purpose from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear conditions for each usage scenario (change, reset, set), but does not explicitly state when not to use this tool or mention alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries full burden. It discloses that notifications are real but marked as test, rate-limited, and verifies pipeline. It does not detail side effects like whether actual users receive the notification, but the test context and rate limit provide adequate transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no fluff. Each clause adds essential information: action, test marking, rate limit, pipeline verification. Perfectly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a test tool with 2 optional parameters and no output schema, the description covers purpose, rate limit, and testing nature. It lacks response format or error handling, but the simplicity makes it adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so schema already documents parameters. The description adds value by stating defaults (source defaults to 'platform', event_type defaults to gateway sample), which aids agent understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool triggers a real test notification, marks audit row with is_test=true, and verifies the full pipeline. The verb 'trigger' and resource 'test notification' are specific, and the details distinguish it from siblings like create_notification_rule or list_notifications.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions rate-limiting (1/min per wallet) and end-to-end verification, implying use for testing. However, it does not explicitly state when to use or not use this tool compared to alternatives, leaving some ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states 'without re-deploying code' indicating non-destructive behavior, but lacks details on permissions, side effects on existing runs, or error states. Adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states purpose concisely, second provides key usage guidance. Every sentence earns its place with no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 5 parameters and no output schema, the description covers the core functionality and usage context well. However, it omits the return value or confirmation of update, which is a minor gap. Sibling tools provide context, but a mention of what the tool returns would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with descriptions for all parameters. The description adds context about the schedule parameter being legacy and advises against using it for new triggers, which adds value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and the resource 'function's timeout or memory', and distinguishes it from deploying code or other function updates. It also contrasts with legacy schedule mutation and directs to ReleaseSpec for new triggers.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use (update timeout/memory without re-deploying) and when not to use (new schedule/email triggers should go through ReleaseSpec), providing clear guidance and alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses non-obvious behaviors: sets archived_at, no-op when archived, admin-only restriction. Lacks details on destruction scope or rate limits, but with no annotations, this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three efficiently written sentences, front-loaded with key purpose. No redundant or missing words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers all essential context for this simple mutation: side effects, no-op condition, admin requirement. No output schema needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and description adds value by explaining the purpose of 'reason' (audit log) and the effect of 'project_id', beyond schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'archive a single project' with specific SQL effect, and distinguishes from sibling tools by noting independence from organization-level lifecycle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Mentions 'Platform-admin only' and no-op behavior, providing clear context. Does not explicitly list exclusions or alternatives, but the context is sufficient for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description fully carries the burden. It discloses atomic ownership flip, return of project keys, persistence to local keystore, and the symmetric behavior with `accept`. Additionally, it mentions the API endpoint. All behavioral traits are transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise paragraph of ~100 words with front-loaded purpose, then conditions, then behavioral details. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool of this complexity (email transfer claim, org handling, key persistence), the description covers all essential aspects: condition, org options, atomic behavior, return value, and API endpoint. No output schema, but return value is mentioned.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description does not add significant extra meaning for parameters beyond what the schema already provides. It only briefly mentions `org_id` context but otherwise replicates schema info.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'Claim' and resource 'incoming EMAIL transfer', distinguishing it from `accept_project_transfer` by specifying it's the email analog. The scope and purpose are unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit conditions for use (email match, org ownership) and explains when to provide or omit `org_id`. It contrasts with `accept_project_transfer` but does not discuss when to avoid using the tool or list alternative tools beyond the analog.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses key behaviors: synchronous return of contract_address before confirmation (no polling needed), pricing details, and that bytecode must be pre-compiled. Missing are authorization requirements (e.g., KMS signer existence) and failure behavior, but overall it provides adequate transparency for most use cases.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (three sentences) and well-structured. It front-loads the primary action and progressively adds details. Every sentence serves a purpose: action+mechanism, bytecode specification, and return behavior+pricing. No redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description adequately explains the return value (contract_address). It also mentions pricing and deterministic address. However, it could be more complete by mentioning error conditions, confirmation status, or required permissions. For a deployment tool, it covers essential aspects but lacks some edge-case details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description adds significant value beyond schema fields. For 'bytecode', it explains it is full creation calldata including constructor args, must be 0x-prefixed hex, and size limit. It clarifies 'value' is optional native token in wei, and 'idempotency_key' behavior. This helps the agent understand parameter semantics deeply.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Deploy a smart contract from a KMS signer'. It details the mechanism (signs a contract-creation tx) and specifies that it returns the deterministic CREATE address. It also distinguishes itself by noting it does NOT compile Solidity and has same pricing as contract_call.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool (to deploy a smart contract) and provides context for bytecode preparation (client-side concatenation, run402 does not compile). It mentions same pricing as contract_call, implicitly guiding against using contract_call for deployment. However, it lacks explicit when-not-to-use guidance or alternative tools beyond the pricing reference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully carries the behavioral disclosure burden. It explicitly states the tool is read-only, does not fetch bytes, purge cache, mutate deploy state, or expose internal CAS URLs. It also lists what the diagnostic includes (match, body status, manifest metadata, structured warnings), providing comprehensive transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured paragraph of three sentences. It is concise, front-loaded with the core purpose, and every sentence adds value without redundancy. Ideal length for quick comprehension.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema and multiple parameters, the description thoroughly explains the tool's behavior, expected diagnostic output (match, status, metadata, warnings), and what it does not do. It covers all necessary context for an agent to decide when and how to use it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all 5 parameters. The description adds context that the tool works for URL or host/path pairs and mentions the method, but does not significantly enhance parameter understanding beyond the schema. Baseline of 3 is appropriate as the schema already does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Read-only authenticated diagnostics for a Run402 public URL or host/path pair' and enumerates specific diagnostic outputs (match, body status, manifest metadata, warnings). It also distinguishes itself by explicitly listing what it does not do, differentiating it from siblings like 'deploy' or 'diagnose_public_url'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the tool's purpose and mentions it provides diagnostics for public URLs or host/path pairs, but does not explicitly state when to avoid this tool in favor of alternatives (e.g., 'diagnose_public_url'). The read-only nature and non-mutating behavior are clear, but exclusion criteria are lacking.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that only live (non-revoked) Telegram bindings are included and lists the fields returned. It does not mention side effects, but as a read-only listing, none are expected.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. The purpose is front-loaded, and the description is structured effectively.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with no parameters. The description covers what is listed, the fields returned, and a primary use case. No output schema exists, but the described return values are sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so schema coverage is 100%. The description adds no parameter info, but per calibration rules, 0 parameters gives a baseline of 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists every notification channel (email, webhook, Telegram bindings) and specifies the returned fields. It explicitly differentiates from siblings like list_notification_rules by focusing on channels and mentions a specific use case for create_notification_rule.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description directly tells when to use it ('Use this to find a telegram_binding_id for create_notification_rule'). It does not explicitly state when not to use, but the purpose is so clear that alternative usage contexts are evident.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, description fully discloses mutation (PATCH), authorization requirements, preconditions (wallet must be linked to org), and response fields (recovery.status, active_wallet_count, next_actions).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single tightly written paragraph with no unnecessary words. Front-loaded with purpose and key constraints.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter mutation tool with no output schema, the description covers purpose, parameters, behavior, and response fields comprehensively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds some context (wallet must be active, null to clear) but does not provide significant new semantics beyond schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action: set or clear an organization's default payout wallet for tenant priced routes, with HTTP method and path. Distinguishes from siblings by specifying the specific resource and operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Specifies admin/owner-only and step-up gating, along with conditions for wallet_address (must be active wallet or null). Provides clear context but does not explicitly compare with alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that values are write-only and injected as process.env, and that setting an existing key overwrites it. Without annotations, this covers the key behavioral traits for a secret-setting operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences with no redundancy. Each sentence adds essential information: what it does, behavioral notes, and usage guidance. Perfectly front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple set tool, the description covers the purpose, behavior, and workflow context (before deploy). No output schema needed for a set operation; the description is complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema coverage, the description adds value by explaining the write-only nature and the overwrite behavior, which goes beyond the schema descriptions. The example also helps understanding parameter formats.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Set a project secret' with an example (STRIPE_SECRET_KEY), and explains that it overwrites existing keys. This distinguishes it from related tools like delete_secret or list_secrets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit says 'Use this before deploy' and advises to 'declare the key with secrets.require' afterwards, providing good context. However, it does not mention scenarios where this tool should not be used or alternative approaches.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Lists four atomic effects (ownership flip, CI revocation, notifications, advisory), explains secret inheritance and GitHub ownership exclusion. Fully discloses behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with summary sentence, followed by structured list of effects, then clarifications. No redundant text. Each sentence adds necessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers preconditions, side effects, and exclusions. Missing return value format, but given the tool is a void-like mutation with no output schema, the description is nearly complete. Slight gap in error scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Single parameter with 100% schema coverage. Description adds no additional semantic meaning beyond the schema's own parameter description, which already repeats the tool's atomic effects. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb ('accept') and resource ('incoming WALLET transfer'). Distinguishes from email transfers via claim_project_transfer and from siblings like cancel_project_transfer, initiate_project_transfer. Includes endpoint and version constraint.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: wallet must equal to_wallet. Explicitly excludes email transfers, directing to claim_project_transfer. Provides clear context for correct invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses ownership, tier limitation, response fields (tier, lease times), step-up gating, and potential error code. With no annotations, this fully covers behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words, front-loaded with the main action, followed by important constraints and response details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter, the description covers creation, inputs, outputs, and a possible error. Could mention that step-up gating is a prerequisite, but overall adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% coverage with a good description. The tool description adds extra context (non-unique, not an id, no tier input), enhancing understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it creates an empty organization on the prototype tier, with the user becoming owner. Specifies the endpoint and uniquely identifies the action among many sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context that it creates on prototype tier, accepts optional display name, and mentions a possible error cap. Doesn't explicitly contrast with alternatives, but is sufficient for the single creation tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It details the full destructive cascade (drop schema, delete functions, release subdomains, tombstone mailbox, remove sender domain, wipe secrets and app versions, set status=purged, remove from local key store) and explicitly states the action is irreversible. This provides comprehensive behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single paragraph that front-loads the key action and details. While it is informative, it is somewhat verbose (over 50 words) and could be more concise without losing essential information. It earns a 4 for being well-structured but slightly wordy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one parameter, no output schema, and no annotations, the description is remarkably complete. It explains the effects thoroughly, distinguishes from related actions, and provides all necessary context for an AI agent to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter `project_id` has a schema description already stating it's the project ID for irreversible deletion. The tool description reiterates this but does not add additional semantic information beyond what the input schema provides. With 100% schema coverage, the description adds minimal extra value for parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Immediately and irreversibly delete a project' and enumerates specific destructive operations (drop tenant schema, delete Lambda functions, etc.). It distinguishes itself from the automatic lease-expiry grace window and mentions the alternative tool `set_tier`, making its purpose unambiguous and distinct from sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool (explicit purge) and when not to (to recover from a missed renewal, use `set_tier`). It also clarifies that this action is distinct from the lease-expiry grace window, providing clear usage context and alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden. It describes the listing behavior and pagination, and implies it's a read-only operation. It could mention that no side effects occur, but the context suffices.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states purpose and output, second explains usage and parameters. No wasted words, front-loaded with essential info.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description mentions the return fields and pagination. Parameter behavior is explained. It is sufficiently complete for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, but the description adds context beyond the schema, explaining that limit bounds results and cursor enables pagination. This helps the agent understand the parameters' roles in the gateway interaction.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists recent deploy operations for a project, specifying the return fields (operation_id, status, release_id, timestamps). It distinguishes from siblings by suggesting use to feed deploy_resume and deploy_events.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit use cases (build deploy-history UIs, find operation_id for other tools) and explains how to use limit and cursor. It does not explicitly state when not to use, but the guidance is clear enough for an agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully carries the transparency burden. It discloses read-only nature, auth requirements ('own-project service-key auth'), cursor opacity, quality tier implications, and behavioral traits like verdict computation and promote-watch semantics. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is lengthy but well-structured, front-loading the core purpose and verdict-first approach. Every sentence provides valuable context, though it could be slightly more concise by grouping related behaviors. The length is justified by the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema, the description thoroughly explains return values (verdict, errors, fingerprints, samples, fetch_logs commands), pagination, filtering, and edge cases (e.g., '0 errors over 0 invocations is absence of signal'). Covers error codes and auth. Complete for a tool with 10 parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, but the description adds significant contextual meaning beyond schema fields. For example, it explains the promote-watch use case for 'new_in', details about 'fingerprint_id' fetching full detail, and implications of 'kind' enums. This raises the score above baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Grouped, release-baselined error memory for a project, verdict-first...' It specifies the resource (errors), action (list), and unique approach (verdict-first with promote vs revert). It distinguishes from sibling tools like get_function_logs by focusing on error identities and verdicts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance, e.g., 'right after an apply/promote activates a release, call errors_list with new_in...' and explains when to inspect new fingerprints. However, it does not explicitly state when not to use this tool or mention alternatives beyond the implicit contrast with other tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but the description implies a read-only, non-destructive operation by stating 'Get tier pricing'. It also discloses that it is free and requires no authentication. However, it does not explicitly state that it does not modify state or mention rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is front-loaded with the purpose and includes all necessary information without any wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that there is no output schema, the description adequately explains the return values by listing the key data fields. This is sufficient for a simple query tool with no parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters (schema coverage 100%), so the description's role is to explain what the tool returns. It adds value by listing the specific data fields: prices, lease durations, storage limits, and API call limits.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool gets tier pricing for Run402 projects, specifies it's free and no auth required, and lists the specific data returned (prices, lease durations, storage limits, API call limits). This distinguishes it from sibling tools like tier_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates when to use it (free, no auth required, anyone can use), but does not explicitly mention when not to use it or suggest alternatives like tier_status. Nevertheless, the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It states the tool does not apply the manifest, implying no side effects. It also clarifies that migration_sql is not executed. However, it does not explicitly state read-only nature or any authentication/rate limit details. The information provided is good but could be slightly more explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, perfectly concise and front-loaded. The first sentence states the core purpose, the second provides usage guidance and clarifications. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description does not mention return values, but for a validation tool the output can be inferred. It clearly distinguishes from sibling tools like deploy. The guidance on migration_sql and deploy manifest boundaries adds completeness. Minor gap: no mention of what the validation result looks like.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (baseline 3). The description adds meaning by explaining that manifest is the auth/expose manifest used by manifest.json, database.expose, and apply_expose, and that migration_sql is only reference context. This adds value beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool validates an auth/expose manifest without applying it. It specifies the resource (auth/expose manifest) and the action (validate), and distinguishes it from deploy-manifest validation. This gives a specific verb+resource combination that leaves no ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells the agent when to use this tool (auth/expose manifest validation) and when not to (deploy manifest validation, advising to use deploy planning/dry-run surfaces instead). It also clarifies that migration_sql is only reference context and not executed, preventing misuse.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations, but description indicates it's a read-only GET, lists return fields, and implies authentication via wallet. Could explicitly state that no side effects occur, but still clear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no fluff. HTTP method and path are front-loaded. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-param tool with no output schema, it adequately describes return fields (principal fields, authenticator_id, org memberships). Could mention error cases or response format, but not required given simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters, so schema coverage is 100%. Description adds no parameter info, but none needed. Baseline 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it resolves the caller's principal and org memberships, with specific verb and resource. Distinguishes from sibling 'status' by noting remote vs local identity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when to use this vs 'status': remote identity vs local wallet/profile state. No other guidance needed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description covers behavioral traits: no source change, no new release, re-bundles with pinned deps, picks up wrapper fixes, strict opt-in, auth requirements, and error case for unlocked deps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is verbose but well-structured, front-loaded with key action, and each sentence adds value. Could be slightly more concise but appropriate for complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers action, preconditions, error conditions, and related tools, but does not describe return values since no output schema exists. Comprehensive given tool simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter descriptions. Description adds significant context for 'name' (optionality, batch behavior) beyond what the schema provides, adding clear semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool refreshes functions onto the current wrapper/bundled runtime without changing source, distinguishing it from redeployment and sibling tools like deploy_function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use (wrapper fixes), when not to (unlocked deps, suggesting deploy_function), and provides context (opt-in, wallet-authed, billing grace allowed). Also points to list_functions to find stale functions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses multiple behavioral traits: 72h expiry, freeze of owner mutations, no refund of tier lease, no GitHub transfer, billing policy migrate, and immediate completion for org. Minor missing details like what happens on expiry or cancellation, but overall strong.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single dense paragraph with clear logical flow: purpose, recipient options, prerequisites, side effects, and version notes. Every sentence adds value, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 8 parameters, no output schema, and no annotations, the description covers purpose, usage, behavior, and parameter semantics comprehensively. It explains all three recipient paths and their consequences. Could mention error handling or expiry behavior in more detail, but overall quite complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3, but the description adds significant value beyond the schema: it explains the exclusivity of the three recipient parameters, the pending vs immediate behavior, default billing policy, and the purpose of retain_collaborator_role for email rail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool initiates a project transfer and specifies three distinct recipient types (wallet, email, org), each with its own completion path. It distinguishes from sibling tools like accept_project_transfer and claim_project_transfer by describing the flow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use each recipient type, prerequisites (must own/admin project, for org must own both orgs), and the behavioral differences (pending row with 72h expiry for wallet/email, immediate completion for org). It contrasts with sibling tools named in the context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: read-only, durable cursor, expired cursor returns reset:true + earliest_cursor, works on frozen projects. It also explains the nature of the feed (app vs platform events) and next_actions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is somewhat lengthy but well-structured with front-loaded main purpose. Every sentence adds value, though some details could be condensed. It effectively uses examples and conditional logic.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (6 parameters, no output schema), the description is remarkably complete. It explains the return structure (cursor, events with next_actions), pagination, filtering, and use cases. No gaps for effective usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but description adds significant meaning: explains the enum values for source, how event_type composes with source, that cursor is opaque and how to handle it, and the relationship between project_id and org_id. This goes well beyond schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it is a 'durable, cursored feed' of project events, including specific event types like deploy activations, mailbox suspensions, etc. It distinguishes from sibling tools by emphasizing the cursored feed and filtering capabilities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use guidance: 'Reach for this after any deploy' and 'at the start of a session on an existing project.' Also explains cursor handling and that it is read-only, but does not mention alternatives or when not to use it compared to other event-related tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that no authentication is required, the endpoint is a public GET /health, and returns per-dependency check results. Without annotations, this provides adequate transparency, though the dependency list is not specified.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, zero wasted words. The first sentence front-loads the primary purpose and differentiation, and the second adds details and alternative usage. Highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of a no-parameter health check, the description is largely complete. It covers what, how, and auth. However, it does not specify the return format or error handling, which could be clarified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With no parameters in the schema, the description adds all necessary meaning by explaining the tool's operation. No parameter documentation is needed, and the description fully compensates for the empty schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it is a liveness check for the Run402 SERVICE, explicitly distinguishing it from organization status checks. It uses a specific verb ('liveness check') and resource ('Run402 SERVICE'), and differentiates from the sibling 'status' tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance on when not to use this tool ('not your organization') and directs to the alternative 'status' tool for organization status. This helps agents choose correctly among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses key behavioral traits: polls until expected SHA or timeout, only for mutable URLs, single vantage point (us-east-1), and on timeout returns isError=true for fallback branching. Since no annotations are provided, this carries full burden and does so thoroughly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded with the main action. It is detailed but every sentence earns its place. Slight verbosity prevents a 5, but it remains efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a polling tool without output schema, the description covers purpose, behavior, constraints, and fallback. It mentions return on timeout but not on success; still adequate given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% but the description adds context beyond the schema, e.g., 'sha256' is 'from a preceding upload' and timeout_ms default. This adds semantic value, justifying a score above baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool polls the CDN until a mutable blob URL serves the expected SHA-256 or timeout. It explicitly distinguishes mutable from immutable URLs, providing a specific verb and resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It specifies when to use the tool: after re-upload to an existing mutable key when end-user visibility is required. It also contrasts with immutable URLs where no waiting is needed, and suggests a fallback to immutableUrl on timeout.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses the irreversible nature ('drops all messages and webhook subscriptions') and the need for confirm=true, which is critical for a destructive action. No annotations present, so description carries full burden and meets it well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no filler. Front-loaded with purpose and warnings. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 parameters, no output schema, and no annotations, the description covers all necessary behavioral and parameter details, including error cases and prerequisites, making it complete for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 100% schema coverage, the description adds significant extra semantics: resolution behavior for omitted mailbox_id, ambiguity error details, and the implication of project scope. This goes beyond the schema's descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete the project's mailbox' with specific verb and resource. It distinguishes from siblings like create_mailbox and update_mailbox by specifying the irreversible deletion of all messages and webhook subscriptions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly requires confirm=true and explains behavior when mailbox_id is omitted, including the ambiguity error for multi-mailbox projects. However, no direct comparison to alternatives or when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully carries the burden of behavioral disclosure. It explicitly states 'Safe to share publicly' and clarifies that this address is for mainnet USDC payments, while request_faucet is testnet-only. These are key behavioral traits that go beyond the bare action of exporting an address.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, each earning its place: the first states the core function, the second adds a safety note, and the third provides critical usage context and a comparison with request_faucet. It is front-loaded and free of fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, no-output-schema tool, this description is complete. It explains what the tool does, why the address is safe to share, and how it fits into the broader payment flow (mainnet vs testnet). No critical information is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline for parameter semantics is 4. The description does not need to add parameter details, and it doesn't attempt to, which is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Export') and resource ('local agent allowance address'), clearly indicating the tool's function. It also distinguishes itself from sibling tools like request_faucet by explaining the mainnet vs testnet contrast, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use this tool (to get a publicly shareable address and as the mainnet on-ramp for real USDC payments) and when not to use it (request_faucet only funds testnet, which cannot settle real payments). This directly addresses usage context and alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It states 'Reads public GET /status. No auth, no allowance required'—a clear, accurate disclosure of read-only, public, non-consuming behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences. First sentence front-loads the tool's purpose; second provides contrast and context. No extraneous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple status check tool with no params and no output schema, the description is complete: it explains scope (service vs org), accessibility (public, no auth), and resource impact (no allowance). Adequate for an agent to select and invoke.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, schema coverage 100%. Baseline for 0 params is 4. Description adds no param info, which is unnecessary; it does mention the HTTP endpoint, but that is not parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Reports on the Run402 SERVICE (availability, capabilities, operator, deployment)', specifying both the verb 'reports' and the resource 'SERVICE'. It distinguishes itself from the sibling 'status' by noting it is not about the organization.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when not to use: 'For your organization status (allowance, tier, projects), use `status`.' Also notes 'No auth, no allowance required', guiding the agent on prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![run402 MCP server](https://glama.ai/mcp/servers/kychee-com/run402/badges/card.svg)](https://glama.ai/mcp/servers/kychee-com/run402)

Score Badge

Copy to your README.md:

[![run402 MCP server](https://glama.ai/mcp/servers/kychee-com/run402/badges/score.svg)](https://glama.ai/mcp/servers/kychee-com/run402)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/kychee-com/run402'

If you have feedback or need assistance with the MCP directory API, please join our Discord server