portkey-admin-mcp

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v1.0.5
Disambiguation4/5
Most tools have distinct purposes due to consistent verb_noun naming and clear descriptions. However, the large number of analytics tools (e.g., get_error_analytics, get_error_rate_analytics, get_error_stacks_analytics) and similar names for get_user_stats, get_users_analytics, get_analytics_group_users cause some potential confusion, though descriptions help differentiate.
Naming Consistency5/5
Nearly all tools follow a consistent verb_noun pattern (e.g., create_api_key, list_workspace_members, update_prompt_version). There are minor exceptions like validate_completion_metadata, but the overall pattern is very regular.
Tool Count2/5
With 150 tools, the server is extremely large. While it covers a broad admin domain, this number exceeds typical coherence thresholds; many tools could be consolidated or the server scope narrowed.
Completeness5/5
The tool surface is remarkably comprehensive, covering full CRUD for users, workspaces, keys, integrations, promts, configs, analytics, MCP, and more. Obvious lifecycle gaps are minimal; the set supports end-to-end admin workflows.
Average 4.4/5 across 150 of 150 tools scored. Lowest: 3.4/5.
See the Tool Scores section below for per-tool breakdowns.
- 1 of 2 community issues answered or closed in the last 6 months
- 98 commits in the last 12 weeks
- Last stable release on August 28, 2026
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
This repository includes a glama.json configuration file.
This server has been verified by its author.
View related servers

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readonly, idempotent, and non-destructive behavior. The description adds context on return fields (id, name, slug, provider, status, etc.) but doesn't elaborate on pagination details or authentication requirements, which are partially covered by the schema. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences that are front-loaded with purpose and filters, followed by a use case and return information. Every word adds value, no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the rich annotations, complete schema, and presence of an output schema, the description effectively covers purpose, parameters, return fields, and usage context. Minor omission: no mention of pagination behavior beyond schema, but that is documented elsewhere.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description references 'optional workspace or type filters,' which maps to the 'workspace_id' and 'type' parameters, but adds no new semantic meaning beyond what the schema already provides. No compensation needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List org-level AI provider connections with optional workspace or type filters' – a specific verb and resource. It further differentiates from siblings like 'list_integration_models' and 'list_integration_workspaces' by indicating it returns integration details and slugs, which are prerequisites for updates.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises 'Use this to find integration slugs before model or workspace updates,' providing a clear use case. While it doesn't list alternatives or when not to use it, the context implies this is the correct tool for discovering integrations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds value by specifying returned fields and purpose, but does not disclose additional behavioral traits beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (two sentences) and front-loaded with the action. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (1 optional param, annotations present, output schema exists), the description covers the return fields and purpose adequately for agent invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the parameter description is already clear. The description reiterates the optional parameter usage but does not add new semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it lists partials across collections with optional filtering, and mentions return fields. It distinguishes from sibling tools like get_prompt_partial and update_prompt_partial by indicating its role as a list operation for selection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description explains when to use (to list partials and choose an ID for further actions) and mentions optional collection filtering. It doesn't explicitly state when not to use, but provides clear context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate mutation (readOnlyHint false) and non-destructiveness, and the description adds that it creates or updates the target and returns version IDs. However, given openWorldHint true, the description does not disclose potential side effects or required permissions, leaving some behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise—just two sentences—with no extraneous information. It front-loads the primary action and then provides usage guidance, making it efficient and easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's 5 parameters and mutation nature, the description covers core purpose, usage context, and return values. It does not mention error conditions or prerequisites, but the presence of a (presumed) output schema reduces the need for full detail. Overall, it is fairly complete for the use case.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 5 parameters have descriptions in the schema (100% coverage), so the description adds minimal extra parameter-level detail. It mentions return values (version IDs) but not parameter specifics, warranting the baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: copy a prompt between environments and auto-create/update the target. It explicitly distinguishes itself by mentioning staged releases and synchronization without manual edits, which sets it apart from siblings like create_prompt, update_prompt, and migrate_prompt.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear when-to-use context: for staged releases when automated synchronization is desired. However, it does not explicitly state when not to use this tool or compare it to alternatives like migrate_prompt, which is present among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, so the description adds context about immediacy and overriding integration settings. This provides behavioral insight beyond annotations, but it doesn't detail potential side effects, authorization needs, or reversibility. Given annotations cover some aspects, a score of 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loaded with the action, and every sentence provides essential information. No unnecessary words or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with a complex schema and good annotations, the description covers key context: action, effect (immediate override), and prerequisite inspection step. It doesn't document output, but an output schema exists. It might be slightly lacking on preconditions (e.g., server existence) but overall is complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the description doesn't need to elaborate much on parameters. It mentions 'enable or disable capabilities' which aligns with the 'enabled' field, but adds no new meaning beyond the schema. The description does not explain the structure of the capabilities array or enum values, but the schema already documents these.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool enables or disables capabilities on an MCP server, uses a specific verb ('update_mcp_server_capabilities' inherently suggests modification), and distinguishes from siblings like 'update_mcp_integration_capabilities' by noting it overrides integration-level settings. It also advises to use 'list_mcp_server_capabilities' first, further clarifying scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly recommends using 'list_mcp_server_capabilities' first to inspect current capabilities, providing clear when-to-use guidance. It states changes take effect immediately and override integration settings, helping differentiate from other update tools. However, it does not explicitly mention when not to use this tool or list all alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that reset_usage clears counters immediately, adding behavioral context beyond the annotations (which only indicate non-destructive, non-idempotent). It also mentions the return value. This is good but could be more detailed about other side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the main purpose, followed by a critical usage note and return value. Every word is necessary and no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 12 parameters and an existing output schema, the description covers the main categories and return value. It lacks details on authorization or behavior for omitted fields, but the output schema and schema descriptions compensate adequately.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema coverage, the baseline is 3. The description only generally mentions 'metadata, limits, or expiration' and specifically calls out reset_usage, but does not add significant meaning beyond what the schema's parameter descriptions already provide.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates a provider's metadata, limits, or expiration, with a specific verb and resource. While it differentiates from sibling update tools by targeting 'provider', it does not explicitly distinguish from other update tools, but the resource is clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear guidance on when to use the reset_usage parameter ('only when you intend to reset quota tracking'), but does not offer broader when-to-use or when-not-to-use guidance relative to other tools, though the context is sufficient given the tool's specificity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate it's not read-only or destructive. The description adds that rotating the key is immediate and limit changes propagate to downstream prompts/configs, providing useful behavioral context beyond what annotations offer. It builds on the 'openWorldHint' by explaining downstream effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loading the purpose and then adding key effects and return value. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, the description appropriately avoids repeating return details but mentions updated name, slug, and status. It covers the tool's effect on downstream resources, making it fairly complete for an update operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers 100% of parameters with descriptions, so the description's mention of 'name, secret, note, or limits' adds minimal new information. It does not clarify edge cases or constraints beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update a virtual key's name, secret, note, or limits,' using a specific verb ('Update') and resource ('virtual key'). It distinguishes from siblings like create_virtual_key and delete_virtual_key by focusing on modification.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context by noting that key rotation takes effect immediately and limit changes affect downstream, but it does not explicitly state when to use this tool versus alternatives or when not to use it. No exclusions or comparisons are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations (readOnlyHint=false, destructiveHint=false) indicate write operation that is not destructive. Description adds that it returns collection id and slug and does not move prompts, providing useful behavioral context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is concise, one sentence with two clauses. Front-loaded with main purpose. Could be slightly more structured but is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given simple tool with output schema, description covers key aspects: purpose, usage context, return values. No missing critical details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so schema already describes both parameters. Description adds marginal value: it relates 'name' to namespace concept, but doesn't add semantic detail beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'Create' and resource 'prompt collection' with explicit purpose 'organizing prompts by app'. Distinguishes from sibling create_prompt by noting it's a prerequisite.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states 'Use this when you need a new namespace before create_prompt', providing clear context. Does not explicitly state when not to use, but context implies it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate this is a write operation (readOnlyHint=false) and non-destructive (destructiveHint=false). The description adds that it registers the server and returns id/slug, but does not elaborate on potential side effects or additional behavior beyond creation. With minimal annotation context, the description provides some added value but not extensive behavioral detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, consisting of two focused sentences. The first sentence states the core purpose, and the second provides contextual workflow advice. While efficient, the second sentence could be slightly more compact, but no unnecessary information is present.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (stated in context), the description appropriately omits detailed return value documentation. It mentions the returned id and slug. The description also advises on a necessary prerequisite step. For a creation tool with good parameter documentation, this is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for all 4 parameters. The description mentions the required parameters (name, mcp_integration_id) and notes slug and description are optional, but does not add new constraints or semantics beyond the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates an MCP server under an existing integration. It specifies the resource ('MCP server') and action ('Create'), and distinguishes from sibling tools by mentioning the prerequisite of using list_mcp_integrations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: use list_mcp_integrations first to find the parent integration, then use this tool to create the server, and later use capabilities or access tools for configuration. This clearly indicates when to use this tool versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, destructiveHint=false, establishing safety. The description adds value by disclosing the enterprise gating and the 403 error response on failure, which goes beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, each serving a distinct purpose: the first defines the tool's function and output; the second provides usage guidance, sibling distinction, and access constraints. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 29 parameters, 100% schema coverage, output schema, and rich annotations, the description covers core purpose and a key sibling. However, it lacks details on time-series bucket granularity, pagination, or ordering. Still, the structured fields and schema compensate partially.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are already well-documented. The description does not add extra meaning to specific parameters but clarifies the overall output structure (summary fields, per-bucket counts). Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves token-usage time-series data with specific metrics (summary.total_tokens, summary.prompt_tokens, summary.completion_tokens, per-bucket counts). It also distinguishes from the sibling 'get_cost_analytics' by noting the difference between token volume and spend.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use this for consumption trends' and suggests 'use get_cost_analytics when you need spend instead of token volume'. It also notes the enterprise gating and 403 error for non-enterprise plans, providing clear context for when the tool is applicable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, destructiveHint=false, fully disclosing the safe, read-only, idempotent nature. The description adds the specific return fields (ids, names, colors, status, timestamps) and the purpose for downstream usage, but does not reveal behavioral traits beyond what annotations already convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states purpose and scope, second lists return fields and links to related tools. No extraneous words, front-loaded with key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Description covers purpose, use case (choosing label_id), return fields, and optional filters. With comprehensive annotations (readOnly etc.) and a fully described schema, the description is complete and informative for an agent to invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with all 5 parameters already described in the input schema. The description only adds 'optional search and scope filters' as a summary, which does not materially enhance understanding of individual parameters. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'List' and resource 'labels', specifies scope (workspace/organisation) and optional filters. It explicitly distinguishes from sibling tools like get_prompt_label and update_prompt_version by explaining the return fields and downstream use.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description provides clear context: it can be used to obtain a label_id for subsequent operations (get_prompt_label, update_prompt_version). It mentions optional search and scope filters but does not explicitly exclude alternatives or provide when-not-to-use scenarios, which is acceptable for a list tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover safety (readOnlyHint, destructiveHint, idempotentHint). Description adds that it returns specific fields but does not mention ordering or limits. Adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states purpose and output, second gives usage guidance. No unnecessary words, well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With output schema present, description appropriately lists return fields. Usage guidance is complete for a simple list tool. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% coverage with description for prompt_id. Description does not add extra semantics beyond the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists all versions of one prompt, specifying the fields returned (version number, description, status, label, template preview). It distinguishes from siblings like list_prompts and get_prompt_version.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly suggests use for history or choosing a version_id before publish_prompt or update_prompt_version. Provides clear context but does not explicitly exclude alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare this as a safe, read-only, non-destructive, idempotent operation. The description adds value by listing the return fields (total, name, slug, status, usage limits, etc.), but does not disclose additional behavioral traits like pagination behavior beyond the schema parameters or authentication requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: the first specifies the action and resource, the second provides usage guidance and return details. No unnecessary words, every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with complete schema coverage and an output schema, the description adequately covers purpose, usage context, and key return information. It is sufficient for an agent to understand when and how to use it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already fully documents the two parameters (page_size, current_page). The description does not add meaning beyond the schema's descriptions, qualifying for the baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists provider API keys stored as virtual keys, specifies the context (Portkey org), and directly addresses a common use case (finding slugs). It effectively distinguishes itself from sibling tools like list_api_keys or get_virtual_key by naming the resource type and purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises when to use the tool ('to find slugs before wiring prompts/configs or auditing limits'). It provides clear context but does not explicitly state when not to use it or name alternative tools for different scenarios (e.g., retrieving a single key).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=false (write operation) and destructiveHint=false. The description adds behavioral context: 'the new version becomes the policy anchor for downstream use', implying versioning and dependency. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core purpose. Every sentence adds value: first defines the tool, second explains the usage pattern. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the essential aspects: creation, contents (checks and actions), and lifecycle hint. Given the tool's complexity (nested objects, output schema), it provides sufficient context for an agent to use it appropriately. Could mention required fields but schema already handles that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description does not add extra meaning beyond what the schema already provides for parameters like 'checks' and 'actions'. It focuses on purpose rather than parameter details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a guardrail with checks and actions for request filtering', providing a specific verb and resource. It distinguishes from 'update_guardrail' and 'delete_guardrail' among siblings. The phrase 'policy anchor for downstream use' adds further context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises 'Create it first, then reference it from configs', giving a clear usage pattern. It does not mention alternatives or when not to use, but the context is sufficient for an agent to infer the appropriate scenario.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide basic hints (readOnlyHint=false), and the description adds behavioral context: returns specific IDs (partial id, slug, version id) and notes the version remains inactive until published. This goes beyond annotations without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. The verb and resource are front-loaded, and the key return values and state are succinctly covered.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema, the description appropriately focuses on key return fields and version state. It omits details like workspace_id requirements for org-level keys, but overall it is sufficiently complete for a creation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% coverage with descriptions for all 4 parameters. The description does not add extra meaning beyond the schema, so baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create') and the resource ('reusable prompt partial'), and explains its purpose for shared snippets/macros. It distinguishes itself from siblings like 'create_prompt' by emphasizing reusability and the inactive status until publication.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear use case ('for shared snippets or macros') and mentions that the version stays inactive until published, which guides when to use. However, it does not explicitly state when not to use this tool or mention alternatives like 'publish_partial' for activation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate a write operation (readOnlyHint=false) that is not destructive (destructiveHint=false). The description adds behavioral detail that the provider inherits the integration key but enforces limits and expiration independently per workspace, providing nuance beyond annotations. It could mention error conditions like duplicate names.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences efficiently cover purpose, behavioral context, and return value with no wasted words. Very concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 12 parameters and the presence of an output schema, the description adequately covers creation and the return of id and slug. It could mention the prerequisite of an existing integration, but overall it is complete for a creation tool with annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, providing a baseline of 3. The description adds context about how integration_id and workspace_id interact (inherits key but independent limits), which adds value beyond individual parameter descriptions. No redundancy.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Create' and the resource 'workspace provider backed by an org integration', distinguishing it from sibling tools like create_integration and update_provider. It also mentions the return value.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when a provider tied to an integration is needed, but does not explicitly state when to use or not use this tool versus siblings like create_integration or create_virtual_key. No exclusions or alternatives are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide basic behavioral cues (readOnlyHint false, destructiveHint false, idempotentHint false, openWorldHint true). The description adds no further behavioral context beyond what is already in annotations, but does not contradict them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with an additional sentence for alternative guidance. It is concise, front-loaded, and contains no redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (8 parameters, 5 required) and the presence of an output schema, the description is mostly complete. It could mention the possibility of duplicate creation, but the annotations and sibling tools provide sufficient context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% description coverage for all 8 parameters, so the description adds no additional meaning beyond listing some parameter names and noting that conditions and group_by are required, which is already in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a request or token throttle, listing key parameters. It distinguishes itself from the sibling tool create_usage_limit, which is used for cumulative budgets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use this tool (for rate limiting) and when not (use usage limits for cumulative budgets). Also notes that conditions and group_by are required.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that authentication credentials are sent to Portkey and exposed in the transcript, and clarifies that the return value is a UUID and slug, not the actual secret. This adds value beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tightly written sentences with no filler; front-loaded with purpose and followed by important behavioral notes.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (11 parameters, auth patterns) and the existence of an output schema, the description covers the essentials: creation, security implications, and return type. Could mention the relationship between manager_type and auth_config, but schema descriptions compensate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters with descriptions; the tool description does not add new parameter-level details beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates a reference to external secrets from three named providers and explicitly distinguishes itself from other tools by mentioning it never returns the resolved secret.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides security guidance to prefer short-lived or workload identity modes, but does not explicitly compare against sibling tools like update_secret_reference or delete_secret_reference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds context beyond annotations: cumulative budget, optional alerts, weekly/monthly resets. Annotations already show it's a write operation (readOnlyHint false) and not destructive, so the description complements without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words. The first sentence conveys purpose and key features; the second adds usage guidance and an alternative. Info is front-loaded and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (9 params, output schema exists), the description covers the core purpose, required fields, and alternative. It omits explicit mention of return value or scoping parameters, but the schema handles those. Slightly better could mention that it creates a usage limit entity with an ID.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with detailed descriptions for each parameter. The description adds little beyond stating that conditions and group_by are required, which is already in the schema. It provides no extra format or examples, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates a cumulative budget for cost or tokens, with optional alerts and resets. It distinguishes itself from the sibling 'create_rate_limit' by specifying that rate limits are for request throttling, not budgets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'use rate limits when you need request throttling,' providing a clear alternative. It also notes that 'conditions and group_by are required,' guiding the agent on mandatory fields. However, it could further clarify scenarios like per-user vs per-key limits.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide safety hints, but description adds value: encryption, one-time return of raw key, immediate application of limits. These behaviors are not in annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences, action first, no wasted words. Efficiently communicates key facts.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With output schema present, return value explanation is sufficient. Description covers security, slug usage, and immediate limit enforcement. Minor gap: no error handling info, but completeness is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, baseline 3. Description clarifies that optional usage/rate limits apply immediately, adding behavioral context beyond schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool stores a provider API key as a virtual key, specifying encryption, slug return, and optional limits. It distinguishes from siblings like create_api_key by focusing on virtual keys and encryption.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises to save the returned slug for use in prompts/configs, which is helpful but lacks explicit when-to-use vs alternatives (e.g., create_api_key). No exclusion or alternative guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate non-read-only, non-destructive, non-idempotent behavior. Description adds return value info (id, name, slug) and slug auto-generation, but does not disclose additional behavioral traits beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. First sentence states purpose, second provides key behavioral detail. Information is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters (1 required), nested objects, and no output schema, the description covers creation purpose, slug behavior, and return fields. Schema covers the rest. Could briefly mention metadata or default usage but not necessary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% description coverage, so baseline is 3. Description adds value by explaining slug auto-generation behavior and mentioning return fields, which clarifies parameter usage beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Create a workspace to isolate resources, API keys, and team members', specifying the verb and resource. Differentiates from siblings like create_api_key or create_collection by naming the workspace entity and its isolation purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explains slug auto-generation behavior ('If slug is omitted it is auto-generated from the name'), which guides usage. However, does not explicitly mention when to use vs. alternatives like update_workspace or delete_workspace.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate destructiveHint=true. The description adds context about irreversibility and the effect on prompts (lose grouping), going beyond annotations. It does not contradict any annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: first states the purpose, second adds behavioral context. Every sentence adds value, and it is front-loaded. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the core action and its effects. It is complete for a simple delete operation with one parameter. Could mention the output schema, but not necessary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with a description for collection_id. The tool description does not add additional meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a prompt collection by ID', specifying the verb and resource. It distinguishes itself from siblings like delete_prompt by focusing on collections and their effect on prompt grouping.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides guidance on when to use (delete collection) and warns about consequences ('cannot be undone'). It suggests reassigning prompts first if organization matters, but does not explicitly list alternatives or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already set destructiveHint=true and readOnlyHint=false. The description adds value by specifying irreversibility and cascading effects on virtual keys, providers, and workspace access.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loaded with the primary action, and every sentence adds critical information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter delete operation, the description covers purpose, consequences, and parameter. It is nearly complete, though the output schema is not described.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema coverage, the description adds no extra meaning beyond the schema's explanation of the 'slug' parameter. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete an integration by slug,' providing a specific verb and resource. It distinguishes this tool from siblings like delete_api_key or delete_provider, which target different entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that deletion is irreversible and breaks dependent entities, giving clear context for when to use the tool. However, it does not explicitly mention when not to use it or suggest alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true, but the description adds critical behavioral context: the operation is irreversible and immediately removes connected users' access. This goes beyond the annotation's binary hint. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences long, providing all necessary information without redundancy. The first sentence states the action, and subsequent sentences add essential warnings. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there is an output schema, the description does not need to describe return values. It covers the irreversible nature, immediate user impact, and the caution about workflows. It is sufficient for the agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The parameter 'id' is fully described in the schema ('The MCP server ID or slug to delete'). The description does not add any additional semantic meaning beyond what the schema already provides. With 100% schema coverage, a score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete') and the resource ('MCP server instance'), making the purpose immediately clear. Among many sibling delete tools, the tool name 'delete_mcp_server' combined with the description sufficiently differentiates it.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes an important usage condition: 'should be used only after confirming no workflows depend on the server.' This advises the agent on when it is appropriate to invoke the tool. However, it does not explicitly mention alternatives or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (destructiveHint=true), the description adds that deletion is irreversible and affects dependent prompts. This clarifies the side-effect and non-idempotent nature beyond the structured annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states the action, second adds the critical consequence. No unnecessary words, well front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple one-parameter tool and existence of an output schema (per context signals), the description adequately covers the action and side effects. It is complete for the complexity level.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter with 100% schema coverage. The schema already describes it as 'Prompt partial ID or slug to delete.' The description adds no extra details beyond 'by ID,' so baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Delete a prompt partial by ID.' It specifies the resource and identifier, distinguishing it from siblings like 'update_prompt_partial' and 'create_prompt_partial.'
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the consequence (permanent deletion, prompt render failures) but does not explicitly state when to use this vs. alternatives like updating or disabling. The warning about reference replacement provides practical guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description adds value beyond annotations by noting pagination, the return structure, and that it's enterprise-gated (returns 403 on non-enterprise plans). Annotations already cover readOnly, idempotent, and non-destructive hints, so the description fills in operational behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the main purpose and return fields. Includes a usage tip and a critical restriction (enterprise-gated). No fluff; every sentence serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the large number of parameters (32) and existence of an output schema, the description covers the key aspects: pagination, grouping, required inputs, and enterprise restriction. It could mention that the many filter parameters are for narrowing results, but the schema already handles that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description does not need to elaborate on parameters. It mentions the required metadata_key and time window but adds no new semantic information beyond what the schema provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool retrieves a paginated metadata breakdown grouped by a required metadata_key, listing specific return fields (total_groups, group_count, metadata_groups). It gives concrete use cases (per-environment, per-feature) and distinguishes from siblings by emphasizing the grouping by metadata_key.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use this for custom breakdowns like per-environment or per-feature analysis' and requires passing metadata_key with the time window. No explicit alternatives or when-not-to-use, but the context is clear enough for an AI agent to decide appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, openWorldHint, idempotentHint, destructiveHint. The description adds meaningful context: the tool is enterprise-gated and returns 403 on non-Enterprise plans. This is a behavioral trait beyond what annotations provide, and there is no contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences: first states purpose, second gives usage differentiation, third notes enterprise restriction. Every sentence adds value, no redundancy. Front-loaded with the primary action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given high parameter count (29), rich schema, and presence of output schema, the description covers the main output elements, sibling differentiation, and access restriction. Could mention aggregation granularity or time-series specifics, but output schema likely covers that. Slightly incomplete but mostly sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description does not provide additional parameter-level details beyond the schema, but the schema already documents all 29 parameters. No extra value added for parameters, but baseline is acceptable.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves error-rate time-series data with summary and per-bucket percentages, and explicitly distinguishes from sibling get_error_analytics by noting that tool gives absolute counts. The verb-resource combination is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context: use for reliability and SLA trends, and suggests an alternative (get_error_analytics) for absolute error counts. Also notes enterprise gating and potential 403 error. Could add more explicit when-not-to-use scenarios, but the sibling comparison is strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only and idempotent behavior. The description adds important access control information: 'Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.' This goes beyond annotations without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only three sentences, each providing essential information: purpose, usage comparison, and enterprise restriction. No redundant or irrelevant content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 29-parameter schema and output schema, the description is sufficiently complete. It explains the aggregation and grouping, and the alternative tool. Minor lack of detail about the output structure is mitigated by the output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the schema descriptions are detailed. The tool description does not add additional meaning to parameters beyond the schema, so it meets the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool's purpose: 'Get stacked error-series data grouped by HTTP status code over time, with summary and per-code series.' It distinguishes from the sibling tool get_error_status_codes_analytics by explaining when to use each.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly contrasts with get_error_status_codes_analytics and mentions enterprise gating and 403 return, providing clear usage context. However, it lacks broader guidance on when not to use this tool relative to other sibling analytics tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false. Description adds useful context about returned fields (masked key, workspace access, etc.) and enterprise restriction. No contradiction with annotations; additional transparency provided.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, each earning its place: purpose+details, usage context, enterprise restriction, error note. Front-loaded and concise with zero waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, usage, enterprise gating, and error codes. Has output schema, so return values not needed. Complete for a simple retrieval tool; could mention authentication but annotations cover readOnly. Sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% coverage for the single parameter 'slug' with a complete description. The tool description only repeats 'by slug' without adding format or source details. Baseline 3 appropriate as description adds no extra value beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'Fetch' and specific resource 'one integration by slug', listing included data (masked key, workspace access, allowed models, configuration metadata). Differentiates from siblings like list_integrations, create_integration, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises use 'before editing provider-specific settings or auditing access', providing clear context. Also warns about Enterprise gating and 403 error, implying when not to use. No alternative tool named, but guidance is strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, destructiveHint, idempotentHint, openWorldHint, so safety is clear. Description adds no new behavioral traits beyond return content, which is expected for a fetch tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, then usage guidance. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given output schema exists and annotations are rich, the description is sufficient for a simple fetch tool. Minor omission: optional filter parameters not elaborated, but schema covers them.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so description adds no meaning beyond what schema already provides for parameters. It mentions needing label_id but that is already required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'Fetch' and the resource 'one label's full definition', and lists included fields (scope, color, status). It explicitly distinguishes from sibling list_prompt_labels.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use this tool (when you already know the label_id) and when to use the alternative (list_prompt_labels for browsing).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds value beyond the annotations by specifying what the tool returns ('the stored string plus version metadata') and its purpose (pre-embedding/update check). Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, so safety is clear. The description does not contradict any annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two sentences, both front-loaded with key information. The first sentence states the core action and objective. The second sentence provides usage guidance and return details. Every phrase earns its place; there is no wasted text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has one required parameter and an output schema. The description covers the main use case (fetching content and version details) and advises when to use it. Given the low complexity and the presence of an output schema, the description is sufficient. It could briefly mention that the partial is from a prompt_template, but that is not necessary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter (prompt_partial_id) with a description already provided ('Prompt partial ID or slug to retrieve'). Schema description coverage is 100%. The tool description does not add additional meaning beyond what the schema already provides, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Fetch a partial's content and current version details') and specifies the resource (a partial). It distinguishes itself from siblings like list_prompt_partials (which lists all partials) and get_prompt (which retrieves a prompt). The verb 'Fetch' is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises when to use this tool: 'Use this before embedding, updating, or checking what {{> partial_name}} resolves to'. This provides clear context for invocation. It does not explicitly mention alternatives, but the context implies that for listing all partials one would use list_prompt_partials, and for mutations one would use create/update_prompt_partial.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, openWorldHint=true, idempotentHint=true, and destructiveHint=false. The description adds enterprise gating and a 403 error response for non-Enterprise plans, which is valuable behavioral context not covered by annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences totaling ~40 words. The first sentence states the purpose, the second provides usage guidance and a critical constraint. No redundant or extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 29 parameters and an output schema, the description provides essential context: what it does, when to use it, and a key authorization constraint. The output schema handles return value details. Acceptable completeness for a tool with rich structured metadata.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for all 29 parameters, so the baseline is 3. The tool description adds no additional parameter meaning beyond the high-level purpose. It does not elaborate on any specific parameters, but the schema fully documents them.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves rescued-request time-series data for requests recovered by retry or fallback handling. It uses a specific verb ('Get') and specifies the resource ('rescued-requests analytics'). This distinguishes it from sibling analytics tools like get_cost_analytics or get_error_analytics.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'Use this only when your configs include resilience features, and use it to measure how often recovery logic saved requests.' It does not explicitly mention alternatives or when not to use, but the context is clear and helpful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint. The description adds value by detailing the returned fields (total, model ids, display names, enabled state, custom-model markers) and the enterprise-gating restriction. It does not discuss pagination behavior (page_size/current_page parameters) but the schema covers pagination and output schema likely defines structure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is highly concise, consisting of five sentences each serving a purpose: stating the action, providing a use case, listing return fields, noting enterprise restriction, and specifying error behavior. No superfluous words or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has a good annotation set, a full schema, and an output schema, the description adequately explains the purpose, usage context, return fields, and error case. It could mention pagination behavior or ordering, but the output schema likely covers the structure. Overall, it is sufficiently complete for an agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for all three parameters. The description does not provide additional context beyond what the schema already offers for the slug, page_size, or current_page parameters. Therefore, it adds minimal semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with a specific verb 'List' and resource 'models enabled on an integration', clearly distinguishing it from sibling tools like list_integrations (which lists integrations) and update_integration_models (which modifies models). It also provides a concrete use case: 'verify model availability before creating prompts or configs'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies when to use the tool ('before creating prompts or configs') and mentions the enterprise limitation with a 403 error for non-Enterprise plans, indicating when it may not work. However, it does not explicitly compare to alternative tools like update_integration_models for modification, though this is implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, etc. The description adds value by specifying that the tool returns audit-relevant fields (default access mode, override flags, connection status) and its role as a precursor to updates.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences efficiently convey action, resource, return data, and usage guidance without superfluous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's purpose, return content, and usage hint. Pagination is handled by parameters. With an output schema present, the description is adequately complete for a list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema fully describes parameters. The description does not add extra meaning beyond implying the 'id' is an MCP server identifier. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'List per-user access for an MCP server' with a specific verb and resource. It lists returned data (default access mode, override flags, connection status) and distinguishes from siblings by mentioning audit and the update tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly says 'use before update_mcp_server_user_access,' providing explicit when-to-use context. It does not list exclusions or alternatives, but the read-only nature of the tool implies appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds 'including current usage' as a brief output detail but does not elaborate on behavior beyond what annotations convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states purpose, second gives usage guideline. No unnecessary words, well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers purpose and usage context. With an output schema present, it is mostly complete, though could clarify what 'entities' refers to (e.g., users, workspaces) or mention pagination.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with one parameter (limit_id) described as 'Usage limit policy ID'. The description does not add additional semantics or examples beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists entities tracked against a usage limit, with current usage. It distinguishes from siblings like list_usage_limits (lists limits, not entities) and names related tools (reset_usage_limit_entity, delete_usage_limit).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: 'Use this to see who is near or over budget before reset_usage_limit_entity or delete_usage_limit.' Provides clear context for usage and avoids ambiguity with sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructive nature (destructiveHint=true). The description adds that it changes accumulated usage for that entity only, which is useful beyond annotations. It could mention reversibility but is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core action, no wasted words. Highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the annotations and output schema existence, the description adequately covers the action and usage hint. It does not explain the exact reset value, but the context of a mutation tool with destructiveHint makes it sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% description coverage for both parameters, so the description does not need to add new meaning. The description reinforces context but does not enhance parameter semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Reset tracked usage for one entity under a usage limit' with specific verb and resource. It distinguishes from sibling tools like 'list_usage_limit_entities' by mentioning it changes accumulated usage for that entity only.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises using 'list_usage_limit_entities' to confirm the target first, providing clear context and a prerequisite step. It does not mention when not to use, but the guidance is sufficient for a destructive tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that updating creates a new version that takes effect immediately for dependent configs, and returns specific identifiers. This adds value beyond annotations (which are minimal). However, it does not cover potential side effects like cascading impacts or permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (two sentences) with front-loaded purpose and sibling distinction. Every sentence adds value: first defines the action, second explains versioning and returns. No redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (4 parameters, nested objects), the description covers the main effect (new version, immediate impact) and suggests reviewing list_guardrails. Output schema exists (implicitly), so return values are adequately stated. Missing minor details like validation or permission mentions, but overall sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the description adds little semantic meaning beyond what the schema already provides for parameters. It mentions updating name, checks, or actions, but the schema already details each parameter. The description's focus on return values does not enhance parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates a guardrail's name, checks, or actions, and distinguishes itself from create_guardrail and delete_guardrail. It also specifies the return values (id, slug, version_id), leaving no ambiguity about the tool's purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use the tool (updating guardrails) and advises reviewing list_guardrails first. It mentions creating a new version and immediate effect, but does not explicitly exclude scenarios where update should not be used (e.g., if no changes needed).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description explains idempotency ('re-running with enabled flipped undoes a change') and immediate application for all users, adding value beyond annotations (idempotentHint: true). It does not mention auth or rate limits but is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, front-loaded with the purpose, and includes all essential information without filler. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema (though not shown), so return values are covered. The description covers purpose, parameters, behavior, and prerequisites. It could mention error handling but is largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds context on sourcing IDs but does not significantly expand on the schema's descriptions of 'id' and 'workspaces'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Grant or revoke workspace access to an MCP integration in bulk,' specifying the verb and resource. It distinguishes from sibling tools like list_mcp_integration_workspaces by detailing how to source IDs and the bulk nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit context for when to use this tool (bulk granting/revoking) and gives sourcing guidance from related tools (list_mcp_integrations, list_workspaces, list_mcp_integration_workspaces). It lacks explicit when-not-to-use or alternatives but is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate a write operation with no destructive or idempotent guarantees. The description adds that changes 'take effect immediately' and 'override the default access setting', providing useful behavioral context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: the first states the core action, the second adds a crucial usage hint. No wasted words, front-loaded with purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the action, immediate effect, override behavior, and a recommended precursor call. With an output schema available (as noted in context signals), the description is sufficient for a simple mutation tool, though edge cases like non-existent users are not addressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear parameter descriptions. The description does not add significant new meaning to the parameters beyond reiterating their purpose, so baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('grant or revoke individual user access') and the resource ('MCP server'). It distinguishes itself from sibling tools like list_mcp_server_user_access and update_mcp_server by focusing on per-user access overrides.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance to use list_mcp_server_user_access first for current state, and explains that changes override default access. While it doesn't cover when not to use it, the alternative recommendation is strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds behavioral context beyond annotations by noting that updates take effect immediately and that changing the slug can break URLs, API key references, etc. Annotations already indicate mutation (readOnlyHint=false) but the description enriches this with practical side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: the first clearly states purpose and differentiates from a sibling, the second provides critical behavioral notes. No unnecessary words; front-loaded with essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 params, nested metadata) and the presence of annotations and output schema, the description covers the main use case, side effects, and differentiation. It does not detail return values but the output schema likely handles that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already describes each parameter. The description merely lists the field names without adding new semantic detail. This meets the baseline but does not add significant value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states it updates a workspace's properties by ID, listing specific fields (name, slug, description, default flag, metadata) and contrasts with the sibling tool update_workspace_member, making the purpose distinct and clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a direct comparison with update_workspace_member and warns about slug changes breaking external links. However, it does not explicitly state when to use this tool versus other alternatives like get_workspace for reading, leaving some implicit context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes beyond the annotations (destructiveHint=true) by detailing the irreversible nature and the cascading effect on configs. This adds valuable behavioral context about consequences, complementing the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences. The first sentence states the purpose, and the second provides a critical warning. No unnecessary words, earning its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple nature of a delete operation with one required parameter and an existing output schema, the description covers all necessary context: what it does, how to identify the guardrail, and the irreversible consequences. Complete for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter 'guardrail_id' is already well-described in the schema as 'The guardrail UUID or slug to delete'. The description does not add additional meaning beyond what the schema provides, meeting the baseline for 100% coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Delete' and the resource 'guardrail', specifying identification by 'id or slug'. It distinguishes itself from sibling tools like 'create_guardrail' or 'update_guardrail'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: it warns that the action is irreversible and removes checks from dependent configs, advising to review those first. While it doesn't explicitly list when not to use, the warning implies careful consideration. It offers clear context for usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (destructiveHint=true), the description explains that deletion only applies to custom entries and that success is returned after removal. This adds behavioral context not fully captured by annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words, front-loaded with action and key constraint. Highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with full schema coverage and an output schema, the description covers purpose, usage, behavioral traits, and return behavior completely. Annotations and schema handle the rest.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description does not add additional meaning to the parameters beyond their names and definitions in the schema. Baseline score applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete a custom model from an integration') and distinguishes between custom and built-in models, implying that built-ins should be handled differently. This differentiates it from sibling tools like 'update_integration_models'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use (custom models) and when not to (built-in models should be disabled instead), providing clear usage guidelines. It does not name the alternative tool but implies it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description adds context beyond annotations by stating 'permanent and removes throttling immediately' and the caution about dependencies. Annotations already indicate destructive behavior, so the description adds useful nuance without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two efficient sentences: first states the action and permanence, second provides a critical caution. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (delete by id) and presence of an output schema, the description covers the essential purpose and behavioral implications. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'id' is fully described in the input schema (100% coverage), so the description adds no additional meaning. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action 'Delete a rate limit by id' and emphasizes permanence and immediate effect. Distinguishes from sibling tools like create_rate_limit, get_rate_limit, update_rate_limit, and list_rate_limits.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit caution to 'review dependent configs and virtual keys before deleting', guiding the agent on prerequisites. However, it does not explicitly state when not to use or compare to alternatives like updating.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true and idempotentHint=true; description adds detail about irreversible nature and immediate impact on dependent integrations/virtual keys, providing context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, front-loaded with action and permanence, then dependency warning. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter destructive operation with full annotations and an output schema, description comprehensively covers consequences and prerequisites.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameter with description 'Secret Reference UUID or slug'. Description adds no further parameter meaning beyond what schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Permanently delete a Secret Reference' with specific verb and resource. Distinguishes from sibling delete tools by specifying the entity type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly warns to confirm dependencies before deletion and describes consequences, but does not state when to avoid using or suggest alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (destructiveHint: true), the description provides concrete details: 'permanent and removes the workspace, its members, configs, API keys, and resources.' This fully discloses the consequences.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences: the first states the action, the second the consequences. No unnecessary words, and critical information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter and an output schema (not shown but present), the description fully covers the effect and scope. No gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage (workspace_id described as 'The workspace ID to delete'), the tool description adds no new meaning beyond 'by id', which is redundant. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Delete' and resource 'workspace by id', making the purpose unambiguous. It distinguishes from other delete tools by specifying 'workspace'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool should be used when permanent deletion is intended, but does not explicitly state when to use it over alternatives (e.g., no mention of archiving or deactivation). It also lacks guidance on prerequisites or caution.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the description adds the behavioral trait of not revealing the secret, which is beyond annotations. It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the main purpose, followed by usage guidance. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers what the tool does, what it inspects, and when to use it. With output schema present, return values are not required. Context is complete for a safe read operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a description for the 'id' parameter. The description does not add additional meaning beyond what the schema provides, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Fetch one API key by UUID' with a specific verb and resource, and distinguishes from siblings like list_api_keys by specifying 'one API key' and 'without revealing the secret'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use this to inspect scopes, defaults, limits, expiration, and reset state before changing access,' providing clear context for when to use it, though it does not explicitly mention when not to use it or list alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, idempotentHint, etc. Description adds enterprise gating and output structure (summary and per-bucket). No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences plus enterprise gating note. Front-loaded with purpose and output fields. No redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite 29 parameters and many filtering options, description doesn't mention filtering capabilities. Output schema exists, but for a complex tool, more detail could help. Adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline 3. Description does not add parameter-specific details beyond schema, which is acceptable.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'get cache-effectiveness time-series data' and lists returned fields (hit_rate, total_hits, total_misses, per-bucket data). It distinguishes from sibling get_cache_hit_latency for speed vs hit/miss ratio.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises 'Use this to measure cache effectiveness; use get_cache_hit_latency for speed rather than hit/miss ratio.' Also mentions enterprise gating and 403 response for non-Enterprise plans.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds Enterprise-gating behavior and 403 error response, which is valuable beyond annotations. However, it does not discuss rate limits, pagination, or other behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states the core functionality, second provides usage guidance and enterprise note. No filler, front-loaded with key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the output schema exists (reducing need to describe return values), the description does not mention the extensive filter parameters available in the schema, which is a gap for a tool with 29 parameters. The description is adequate but could be more complete regarding filtering options.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The tool description does not add any additional meaning or context to the parameters, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves 'raw feedback-score distribution time-series data with per-score buckets' and distinguishes itself from get_feedback_weighted_analytics, which handles calibrated scores. The verb 'Get' and resource 'feedback-score distribution' are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when to use this tool ('understand sentiment mix') and when to use the alternative ('get_feedback_weighted_analytics for calibrated scores'). Also notes Enterprise-gating and 403 error on non-Enterprise plans, providing clear context for usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnly, non-destructive, idempotent. The description adds that it returns active version and history, and gives specific advice for multi-message chat prompts, which goes beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states purpose and output, second provides usage guidance and a specific edge case. No redundant information, perfectly front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple one-parameter tool with output schema and annotations, the description fully covers what the tool returns and when to use it, including a notable edge case for multi-message chats.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3. The description does not add additional semantic meaning to the prompt_id parameter beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it fetches a prompt's full definition, active version, and version history. This distinguishes it from sibling tools like get_prompt_version (specific version) and get_prompt_label (label metadata).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly recommends using this tool before updating, publishing, rendering, or copying a prompt. It provides clear when-to-use context and an alternative approach for multi-message chats, though it does not explicitly state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds valuable context beyond annotations by specifying that the response includes 'limits, rate settings, expiration, and reset status,' which informs the agent about the richness of the data. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two concise sentences: the first states the action and included data, the second provides usage guidance. No extraneous words; every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not shown but indicated), the description is complete: it explains the tool's function, included data, and use case. No gaps given the low complexity of a single-resource fetch operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% coverage with descriptions for 'slug' and 'workspace_id'. The description adds the phrase 'by slug', but does not provide additional semantic detail beyond what the schema already states. Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Fetch one provider by slug' – a specific verb+resource. It details the data returned (limits, rate settings, expiration, reset status) and distinguishes from sibling tools like list_providers and update_provider by focusing on a single entity and use case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear usage context: 'Use this to check consumption or audit configuration before updating.' This guides the agent to use it for auditing or pre-update checks. While it doesn't explicitly name alternative tools, the context is sufficient to differentiate from list_providers or update_provider.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds important context beyond annotations: masking behavior for non-system users, returns metadata and masked auth, not the actual secret value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two precise sentences: first states action and identifier, second adds critical behavioral nuance. No redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers retrieval purpose, identification method, return content (metadata + masked auth), and what is NOT returned (secret value). Output schema handles return structure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only parameter 'id' is fully documented in schema (100% coverage). Description reiterates 'UUID or slug' but adds no new semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Retrieve one Secret Reference' with identifier type (UUID or slug), distinguishing it from sibling list/create/update/delete tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies singular retrieval vs list by using 'one', but lacks explicit when-to-use vs alternatives. Context signals and siblings help differentiate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the tool read-only and idempotent. The description adds value by specifying exactly what data is returned (e.g., masked secret, limits), which is not in annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no redundancy. First sentence states purpose, second gives usage context. Efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the single parameter, full schema coverage, and output schema present, the description is complete. It includes usage guidance and return field highlights.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers the single parameter 'slug' with a description. The description's mention of fetching by slug adds little beyond the schema, achieving the baseline for full schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool fetches a virtual key by slug, listing the returned fields (metadata, masked secret, limits, status, model config). It distinguishes itself from sibling getters like get_api_key.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance: 'Use this before updating or to inspect the current configuration.' It does not mention alternatives but gives clear context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds a minimal behavioral fact: it returns a single member and requires both IDs. However, it does not disclose additional behavioral traits such as error handling, rate limits, or authentication requirements. With annotations carrying most of the burden, this description adds only marginal transparency beyond them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the purpose and the second gives usage guidance. It is front-loaded with the action and resource, contains zero filler, and every word earns its place. This is a model of conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple lookup tool, the description is complete. The input schema fully documents the two required parameters, an output schema exists to define the return value, and annotations cover safety. The description adds the critical context about when to use it versus list_workspace_members. There is no missing information that would prevent correct use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% since both parameters have descriptions. The user_id description is detailed (valid UUID, not email, from list_all_users), while workspace_id's is generic. The tool description restates the parameters but does not add new semantic meaning beyond the schema, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Get one workspace member by workspace_id and user_id.' This specific verb + resource combination distinguishes it from sibling tools like list_workspace_members, add_workspace_member, and remove_workspace_member. It also explicitly names list_workspace_members as the alternative for browsing, reinforcing differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance: 'Use this when you already know both IDs; use list_workspace_members to browse the full roster.' This tells the agent exactly when to use this tool and points to the appropriate alternative, making the choice unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnly, openWorld, idempotent, and non-destructive. The description adds value by specifying the return fields (ids, names, slugs, timestamps) and filtering options, which is useful additional context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: two sentences that cover purpose, filtering, return fields, and usage context with no redundant words. Front-loaded with the main action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only list tool with 4 optional parameters and an output schema (not shown but mentioned), the description adequately conveys the return fields and intended use case, making it complete for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description mentions filtering by name or workspace, which aligns with the search and workspace_id parameters, but does not add new information beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list', resource 'prompt collections', scope 'in the workspace', and optional filters. It also distinguishes from sibling tools by mentioning it returns collection_id for use with create_prompt, get_collection, or list_prompts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use this tool: to obtain a collection_id before creating prompts or listing prompts. While it doesn't state when not to use it or name alternatives, the context is sufficient for an agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, so the description carries less burden. It adds the fields returned but does not disclose potential behavioral details like pagination, ordering, or whether all versions are returned at once.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first sentence states what it does, second provides usage guidance. Every sentence earns its place, no fluff, front-loaded with the most important information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given one parameter, read-only operation, and the presence of an output schema (not shown but indicated), the description covers the key aspects: what is listed (versions with fields) and why to use it (audit, compare). It is sufficient for an agent to decide when to invoke this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% coverage with parameter 'slug' described as 'Configuration slug to list versions for'. The tool description does not add additional parameter meaning beyond what is in the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool lists every version of a config with specific fields (version number, config payload, creator, timestamp). It distinguishes itself from siblings like get_config (single version) and list_configs (list configs) by focusing on listing versions of a single config.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly provides when to use: 'Use this to audit history or compare revisions before update_config or delete_config.' This gives clear guidance on context and alternative operations without being verbose.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations (readOnlyHint, idempotentHint, destructiveHint) already cover safety traits. The description adds no new behavioral details beyond stating it lists guardrails with optional filters, which is consistent but not additional disclosure. Moderate value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two efficient sentences, no wasted words. The first sentence states purpose and fields, the second provides usage guidance. Clearly front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given comprehensive annotations, full schema coverage, and presence of an output schema, the description is complete. It specifies output fields, filters, and usage pattern, leaving no essential gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with all parameter descriptions provided. The description mentions optional workspace/org filters but does not add meaning beyond what the schema already conveys. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List guardrails in the org' and mentions specific fields (id, slug, status, ownership) and optional filters. It distinguishes from siblings by advising use before get, update, or delete operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises 'Use this to find IDs and slugs before get_guardrail, update_guardrail, or delete_guardrail,' providing clear when-to-use guidance and implicit when-not-to-use (when you need a single guardrail, use get_guardrail).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only and idempotent behavior. The description adds valuable context by stating that it returns global access mode and per-workspace enablement, and that it should be used before updates. This goes beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences that front-load the purpose and then add return details and usage guidance. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the presence of an output schema, the description explains the return contents (global access mode and per-workspace enablement), the use case (audit/permission review), and ties to the sibling update tool. For a simple list tool with one parameter, this is complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema fully describes the single parameter 'id' with a clear description. The tool description does not add new meaning beyond referencing the integration context. With 100% schema coverage, baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('List which workspaces can access an MCP integration') and specifies the return content (global access mode plus per-workspace enablement). It also distinguishes from the sibling update tool by explicitly recommending use before update_mcp_integration_workspaces.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear guidance: use for audit/permission review and before an update. However, it does not differentiate from the similar sibling tool 'list_integration_workspaces', which could cause confusion. The guidance is good but not exhaustive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false; the description adds that it does not return authentication configuration or resolved secret values, which are important behavioral traits beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose and key constraint, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With annotations, output schema, and parameter descriptions, the description completes the picture by clarifying what is not returned and how to proceed for more detail.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter descriptions. The description reinforces filtering by manager type, tags, or name but adds minimal extra meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool lists secret references without returning sensitive data (auth config or resolved values), and distinguishes itself from get_secret_reference by mentioning use of UUID/slug for detailed retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says what the tool does not return and suggests using get_secret_reference for full details. Provides filtering options (manager type, tags, name) but does not explicitly state when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds the list of returned fields but no extra behavioral context like pagination or error cases.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence plus a usage hint, no fluff, and front-loads the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists and the tool is simple (one optional param), the description fully covers necessary context for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has only one optional parameter (workspace_id) with a clear description. The tool description adds no additional meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists usage limits with specific fields (id, type, credit_limit, etc.), distinguishing it from sibling tools like get_usage_limit or update_usage_limit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises using this before get_usage_limit, update_usage_limit, or delete_usage_limit. However, it does not mention the sibling list_usage_limit_entities tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare read-only, open-world, idempotent, and non-destructive hints. The description adds that it returns specific fields and implies pagination via page_size and current_page, which is consistent with annotations and adds useful context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, then usage guidance. No redundant phrasing; every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given two optional parameters, no required, and an output schema, the description adequately covers purpose, usage, and return fields. No gaps for a list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already describes both parameters (page_size, current_page). The description does not add additional meaning beyond the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List workspaces' and specifies the returned fields (id, name, slug, default settings, timestamps), distinguishing it from sibling tools like get_workspace (returns a single workspace).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states to use this tool to find a workspace_id before several other tools (get_workspace, update_workspace, etc.), providing clear when-to-use context. However, it does not explicitly mention when not to use it, but the context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that it sends a live check and returns success, response time, HTTP status, and any error. Annotations indicate idempotentHint false and openWorldHint true, which the description supports by mentioning external interaction. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. The description is front-loaded with the purpose, followed by usage guidance. Highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple diagnostic tool with one parameter and an output schema, the description covers essential aspects: purpose, when to use, and what to expect in return. Complete given the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the description does not add extra meaning beyond what the schema provides for the 'id' parameter. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Test connectivity to an MCP server' with a specific verb and resource, and the context of diagnosing reachability distinguishes it from sibling tools like get_mcp_server.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises use before configuration changes or when diagnosing reachability, providing clear context. However, it does not explicitly name alternative tools for other use cases, though these can be inferred from sibling names.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already flag destructiveHint, idempotentHint, and openWorldHint. The description adds valuable behavioral context beyond these: global_workspace_access_enabled affects current and future workspaces, override_existing_workspace_access determines replacement behavior, and per-workspace fields override top-level values. It also notes immediate downstream impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, each earning its place: first states purpose and scope, second explains key toggles, third clarifies precedence, fourth gives operational warning. Slightly dense but well structured and front-loaded with the main purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (9 parameters, global vs per-workspace settings), the description covers the critical nuances and includes a caution to review first. An output schema exists, so return values need not be explained. It is complete enough for a knowledgeable agent to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds meaningful interactions between parameters (e.g., precedence rules, override semantics) that are not apparent from individual property descriptions. This elevates the score to 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool controls per-workspace and global access to an integration, enumerating specific features (cost/rate limits, usage resets, provider creation). This distinguishes it from read-only siblings like list_integration_workspaces and the broader update_integration tool, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance to 'Review list_integration_workspaces first' and warns that access and limits change immediately, giving clear context on when and how to use the tool. It stops short of naming alternative tools for exclusions, but the prerequisite sequence is helpful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true and idempotentHint=true, and the description reinforces 'without making changes'. It adds specific context about billing and attribution error checking, which goes beyond the annotations. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the key action, and every word adds value. No unnecessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The explanation covers the purpose, usage context, and behavioral safety. Since the tool has an output schema, the description does not need to explain return values. It is sufficiently complete for a validation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and each parameter has a detailed description. The tool description does not add extra meaning beyond the schema; it only mentions 'validates required fields and values' without specifying which fields. Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as a preflight validator for billing metadata before run_prompt_completion. It uses a specific verb ('validate') and resource ('billing metadata'), and distinguishes itself from the sibling 'run_prompt_completion' by being a read-only check.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use the tool ('before run_prompt_completion') and why ('catch attribution errors before paying for the call'). It implies not to use it when not about to call run_prompt_completion, but does not explicitly name alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate write operation (readOnlyHint false) and non-destructive (destructiveHint false). The description adds behavioral context: 'takes effect immediately' and 'returns the created feedback IDs and status', which go beyond basic annotations. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that communicates the core action, return value, and alternative. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the moderate complexity (4 parameters, one nested object) and presence of an output schema, the description covers purpose, usage, and behavioral impact. It could mention that weight/metadata are optional, but schema already does so. Almost complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (all parameters have descriptions). The description does not add new semantic meaning beyond the schema; it only reinforces the trace_id link already documented. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('create feedback'), resource ('for a trace or request'), and the linking mechanism ('by trace_id'). It explicitly distinguishes itself from the sibling tool 'update_feedback' by mentioning its use case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'use update_feedback when correcting an existing record', which is a clear when-to-use vs when-not-to-use instruction. No other usage guidelines needed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate this is a non-read, non-idempotent, non-destructive operation, and the description aligns with these hints. It adds meaningful context beyond the annotations: secret_mappings resolve protected values at runtime without storing them in the tool call, and workspace_id is needed for org admin keys. It also clarifies that this tool alone does not complete the MCP server setup, implying a multi-step workflow.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences long, front-loads the core purpose, and every sentence carries distinct information: the action, auth handling, prerequisite, and next steps. There is no redundancy or filler, making it highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a creation tool with 9 parameters, nested objects (secret_mappings), and an output schema, the description covers the main workflow, auth security, and the necessary follow-up (create_mcp_server). It does not explain every enum value or optional parameter, but the schema covers those. The description is sufficiently complete for an agent to understand the tool's role and key constraints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, so the baseline is 3. The description adds extra meaning by explaining the relationship between custom_headers and secret_mappings (targeting configurations.custom_headers) and highlighting that secret_mappings avoid storing secrets in the call. It also specifies when workspace_id is required. This goes beyond mere schema enumeration, warranting a 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Create a Portkey integration for an external MCP server URL', which clearly specifies the action and target resource. It also distinguishes itself from the sibling create_mcp_server by stating that after integration creation, one must call create_mcp_server to configure capabilities and access, making it unambiguous which tool does what.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly instructs to use create_mcp_server after creating the integration, which sets the boundary between the two tools. It also notes that organization admin keys normally require workspace_id, providing a prerequisite. However, it does not explicitly enumerate cases where this tool should not be used or contrast it with other integration tools, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds behavioral details beyond annotations: states it cannot be undone, revokes immediately, can break sessions, and returns success after revocation. Annotations only provide destructiveHint=true.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, all information relevant and front-loaded. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the effect and response, but does not mention prerequisites like permissions. For a simple delete, it is still fairly complete given annotations and output schema exist.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of the single parameter with description. Description adds no extra param details beyond mentioning UUID, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description states specific action 'Delete an API key by UUID', clearly distinguishing from siblings like create_api_key and update_api_key.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly mentions permanence, immediate revocation, and potential impact on active sessions. Could be improved by contrasting with alternatives, but the destructive context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so safety is clear. The description adds value by specifying pagination, output format, and enterprise access restriction, which are beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, front-loaded with the core functionality, followed by usage guidance and access restrictions. Every sentence adds unique value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (31 parameters, output schema, nested objects), the description covers key aspects: pagination, output fields, use cases, and access control. It could mention pagination parameters explicitly, but overall it is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with detailed descriptions for each of the 31 parameters. The description does not add parameter-specific meaning beyond what the schema provides, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a paginated per-user breakdown with specific fields (total_groups, group_count, users array). It distinguishes itself from the sibling tool get_users_analytics by noting the latter is for aggregate trends.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use this tool ('billing, audits, or top-consumer analysis') and when to use an alternative ('use get_users_analytics for aggregate active and new user trends'). Also notes enterprise-gating and 403 response for non-Enterprise plans.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly/intent, so burden is low. Description adds Enterprise-gated constraint and error detail. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with purpose, no superfluous text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers return fields, usage context, and access restrictions. Output schema exists. Could mention aggregation granularity but not required.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description doesn't add parameter-level meaning beyond schema, which is acceptable.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Get cost time-series data' with specific fields (summary.total_cost, etc.). Distinguishes from sibling get_token_analytics by monetary vs token focus.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use for spend analysis and spike detection' and 'use get_token_analytics when you need token volume'. Also notes Enterprise gating and 403 response.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, idempotentHint, and destructiveHint=false. The description adds important behavioral context: the tool is enterprise-gated and returns a 403 error on non-enterprise plans. It also describes the output structure (summary.total_errors and per-bucket counts). This adds meaningful value beyond annotations, though it does not cover all potential behaviors (e.g., pagination).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences: first states the purpose, second gives usage guidance with siblings, third warns about enterprise gating. It is front-loaded with the tool's core functionality and uses no unnecessary words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (29 parameters, many legacy/structured pairs) and the presence of an output schema, the description covers the essential context: purpose, usage guidelines, and a critical restriction (enterprise-gated). It does not explain bucket granularity or aggregation details, but the output schema likely covers those. The description is sufficiently complete for an AI agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and each parameter already has a detailed description in the schema. The tool description itself does not add parameter-specific semantics, but with full schema coverage, the baseline of 3 is appropriate. The description does not need to repeat param info.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Get error-count time-series data with summary.total_errors and per-bucket counts', clearly specifying the verb, resource, and output structure. It distinguishes from siblings by naming get_error_rate_analytics, get_error_status_codes_analytics, and get_error_stacks_analytics for different use cases.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states 'Use this for high-level error trends; use get_error_rate_analytics for percentages, or get_error_status_codes_analytics and get_error_stacks_analytics for breakdowns.' It also notes the tool is enterprise-gated and returns 403 on non-enterprise plans, providing clear when-to-use and when-not-to-use guidance with alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, openWorldHint=true, destructiveHint=false. The description adds the enterprise restriction and 403 response, which are beyond annotations. No contradiction. However, additional behavioral details like rate limits or pagination could be mentioned, but given annotation coverage, this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is very concise: two sentences plus a one-sentence enterprise note. It is front-loaded with the core purpose in the first sentence, and every sentence adds value. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (29 parameters, output schema exists), the description adequately covers the tool's purpose, output type (time-series, summary, per-code series), and usage context. The output schema handles return value details, so description does not need to elaborate. The sibling distinction completes the context. Minor gap: could mention that required parameters are time bounds, but schema handles that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description itself does not add any parameter-level details; all parameter semantics are already provided in the input schema. The description's mention of 'time-series data' implies temporal parameters, but the schema explicitly documents them. Hence, no additional value beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool retrieves HTTP error-code distribution time-series data with summary and per-code series. It uses specific verbs ('Get') and resource ('HTTP error-code distribution time-series data'), and explicitly distinguishes from sibling get_error_stacks_analytics by specifying that tool is for stacked/cumulative breakdowns.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance on when to use this tool ('see which codes occur most often') and when to use the alternative ('get_error_stacks_analytics for stacked or cumulative breakdowns'). Also states that it's Enterprise-gated and returns 403 on non-Enterprise plans, which informs usage conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the safety profile is clear. The description adds behavioral context: enterprise-gating and 403 response, which are beyond annotations. It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with no filler: first states purpose, second gives usage alternatives, third mentions enterprise-gating and error. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 29 parameters, 2 required, and an output schema exists, the description provides a high-level overview (time-series, summary, per-bucket counts) and error condition. It covers essential context without needing to detail all parameters, as the schema handles that. Slightly more could be added about optional filtering, but it's sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters sufficiently. The description adds no additional parameter-specific meaning beyond what the schema provides, resulting in a baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Get' and identifies the resource as 'feedback-submission time-series data' with outputs 'summary totals and per-bucket counts'. It also distinguishes this tool from siblings by naming specific alternatives for breakdowns, ensuring clear differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use this tool ('top-level feedback trend view') and when to use alternatives ('get_feedback_models_analytics, get_feedback_scores_analytics, or get_feedback_weighted_analytics for breakdowns'). Also notes enterprise-gating and 403 error for non-Enterprise plans, providing clear usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations include readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds the enterprise gating constraint and the 403 response on non-Enterprise plans, which is behavioral context beyond annotations. No contradictions. A 4 is appropriate as the description enriches transparency but the annotations already cover most behavioral aspects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loaded with the core purpose, and includes the critical usage distinction and gating info. Every sentence adds value with no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema (not shown but indicated as present), so return value details are not needed. The description covers purpose, usage distinction, and access restrictions. It is complete for a read-only analytics tool with 100% schema coverage and strong annotations. Slight gap: it doesn't mention the read-only nature explicitly, but annotations cover that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for all 29 parameters, including legacy/preferred pairs and formatting notes. The description itself does not add parameter-specific information; it only contextualizes the output as time-series data. Baseline 3 is correct because the schema already provides comprehensive parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves weighted feedback-score time-series data. It explicitly distinguishes from the sibling tool get_feedback_scores_analytics by noting it uses the weight recorded at creation for calibrated metrics. The verb 'Get' and resource 'feedback-weighted-analytics' are specific and unique among siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'Use this for calibrated quality metrics; use get_feedback_scores_analytics for the raw unweighted distribution.' It also warns about an enterprise gating mechanism: 'Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.' This clearly communicates when to use and when not to use, with a named alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, idempotent, non-destructive. Description adds enterprise-gating and 403 error response, plus outlines response structure (summary and per-bucket counts). No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each serving a distinct purpose: response fields, usage guidance, enterprise constraint. No redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 29 params and output schema, description covers purpose, usage, and enterprise constraint. Could mention aggregation granularity, but output schema likely covers response shape. Adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so description need not repeat parameter details. Description does not add new parameter semantics beyond implicit date range from required params. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it gets request-volume time-series data with specific fields (summary.total_requests, etc.). Distinguishes from sibling get_error_analytics by specifying use case. Also mentions enterprise gating.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use this for traffic and reliability trends; use get_error_analytics when you only need error counts.' Provides clear guidance on when to use this tool versus an alternative. Also implies enterprise plan required.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, covering safety. The description adds the crucial behavioral detail that the tool is Enterprise-gated and returns 403 on non-Enterprise plans, which is not in annotations. No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only three sentences: purpose, usage guidance with alternative, and access constraint. Every sentence is essential, no redundancy, and the most critical information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the high parameter count (29) and the presence of an output schema, the description suffices by covering purpose, usage context, and access limitations. It does not detail return values (handled by output schema) or every parameter (handled by input schema), but this is appropriate for a tool with rich structured metadata.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, so the schema fully documents all 29 parameters. The tool description does not add any parameter explanations beyond what the schema provides, which is acceptable per the baseline rule for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description specifies 'Get per-user request-count time-series data with counts grouped by user', which clearly states the verb, resource, and grouping. It explicitly distinguishes from the sibling 'get_users_analytics' by contrasting use cases (heavy users vs aggregate trends).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'Use this to find heavy users and traffic concentration; use get_users_analytics for aggregate active and new user trends instead.' It also discloses that the tool is 'Enterprise-gated' and returns 403 on non-Enterprise plans, helping the agent avoid errors.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, openWorldHint, idempotentHint, destructiveHint. Description adds enterprise restriction and 403 behavior, which is useful beyond annotations. However, no mention of pagination or other side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences plus a warning, no extra words. Front-loaded with core purpose. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given output schema exists, description adequately explains key return fields and enterprise gating. For a 29-parameter tool, it covers the essential context without overloading.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description does not add parameter-specific meaning beyond what schema provides; it focuses on return values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves user-growth time-series data with specific fields (summary.total_active_users, summary.total_new_users, per-bucket counts). It also distinguishes from sibling tools by naming alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance on when to use (growth and adoption trends) and when not (per-user traffic, per-user cost/token detail) with named alternatives. Also warns about enterprise gating and 403 error.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint. The description adds valuable behavioral context not covered by annotations, such as the Enterprise-gated access (returns 403 on non-Enterprise plans) and the distinction from population metrics, which clarifies the tool's scope and limitations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is highly concise with three sentences, each serving a distinct purpose: stating the function, differentiating from a sibling, and noting access restrictions. The key information is front-loaded, with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of a rich input schema, annotations, and an output schema, the description provides sufficient context. It covers purpose, usage differentiation, access constraints, and required parameters. However, it could briefly mention that the output is per-user, but the high schema coverage compensates, so it is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% parameter description coverage, so the baseline is 3. The tool description mentions the required time range but does not add new semantic information beyond what is already in the schema. Hence, it provides no additional parameter insights.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns per-user request and cost analytics for a required time range, using specific verbs ('Return') and resource ('per-user request and cost analytics'). It also distinguishes from the sibling tool get_users_analytics by specifying 'This is usage-by-user, not population metrics'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use this tool (for per-user analytics) and when not to (for population metrics, use get_users_analytics). It also provides access constraint information ('Enterprise-gated. Returns 403 on non-Enterprise Portkey plans'), guiding the agent on proper usage and error handling.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate non-read-only, non-destructive, non-idempotent, and open-world. The description adds behavioral context by noting that workspace assignments are effective only after acceptance, which is not captured in annotations. This is valuable transparency beyond the structured fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences that cover purpose, optional features, and important behavioral nuance. Every word adds value, with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, nested objects, output schema exists), the description covers the main purpose, optional capabilities, and key behavioral point about delayed workspace assignment. It is sufficiently complete for an AI agent to understand the tool's role, though it could briefly mention that an invitation is sent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already describes all parameters. The description mentions optional provisioning of workspace access and API key, which aligns with the 'workspaces' and 'workspace_api_key_details' parameters but adds little extra meaning. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool invites a new org user and optionally provisions workspace access and an API key. It distinguishes from sibling tools like add_workspace_member by noting that workspace assignments apply only after acceptance, prompting use of separate tools for follow-ups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states the tool's use case (inviting a new org user) and clarifies when to use alternative tools for workspace changes after acceptance. This provides clear guidance on when and when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate this is read-only and idempotent, so the description's additional disclosure of the Enterprise-gating and 403 error behavior adds value. However, it doesn't mention pagination or other behavioral nuances that might matter for listing jobs, though with output schema present this is less critical.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: the first sentence defines the action, the second provides usage context, and the third states a critical limitation. No wasted words; every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with a simple single-parameter schema and an output schema, the description covers the core purpose, usage, and error condition. The only minor gap is not explicitly mentioning pagination or ordering, but the output schema likely covers return structure. Overall complete for its complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides 100% coverage for the single parameter, workspace_id, with a clear description. The tool description merely reiterates 'in a workspace' without adding further meaning beyond the schema, so it meets the baseline but doesn't exceed it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists log export jobs in a workspace with status, filters, and timestamps. It distinguishes itself from related tools by explicitly positioning it as the way to find an export_id before using other log export operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use guidance: 'Use this to find an export_id before calling get_log_export, start_log_export, cancel_log_export, or download_log_export.' It also discloses a restriction (Enterprise-gated, returns 403 on non-Enterprise plans), which helps the agent decide if this tool is appropriate in context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint, idempotentHint, and destructiveHint false. The description adds value by detailing the returned content (total, tool, resource, prompt surface) but does not explicitly mention pagination behavior. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the primary action, and every word adds value. It is concise without any fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema and annotations cover safety, the description covers purpose, return content, and usage guidance well. Minor missing detail on pagination behavior prevents a perfect score, but overall it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with each parameter described. The description adds marginal value by hinting at pagination via 'total' and mentioning returned fields, but it does not elaborate on parameter syntax or constraints beyond the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists capabilities exposed by an MCP server instance, specifying it returns total plus tool, resource, and prompt surface. It explicitly distinguishes itself from the sibling integration-level capability list, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'use this instead of the integration-level capability list when you need server-specific exposure.' This clarifies when to use this tool and names the alternative (list_mcp_integration_capabilities), offering clear usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate safe, read-only, idempotent behavior. The description adds value by specifying the return fields (pagination summary with id, name, slug, model, status) and the intended use case, without contradicting annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences that are front-loaded with purpose and filters, then return format and use. Every sentence adds value; no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 optional params, output schema present), the description covers scope, filters, return fields, and use case. It does not mention sorting or default pagination, but output schema and API conventions likely cover these.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with good parameter descriptions. The description groups filters ('optional collection, workspace, or search filters') to provide context, though it doesn't mention page_size or current_page. This adds understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the verb 'list' and resource 'prompts' across the workspace, with optional filters. It distinguishes from sibling tools like get_prompt, update_prompt, and render_prompt by stating the purpose: to choose a prompt_id for subsequent operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool (listing prompts with optional filters) and implies that for a single prompt by ID, one would use get_prompt. However, it lacks explicit exclusions or alternative recommendations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, establishing safe, read-only behavior. The description adds value by detailing the specific fields returned (organization role, workspace role, status, timestamps), which goes beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences, front-loading the purpose and providing immediate actionable guidance. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (one parameter, output schema exists), the description is largely complete. It mentions the key fields returned, though it could optionally address pagination or ordering. However, the absence of such details does not significantly hinder completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 100% coverage with one required parameter (workspace_id) and a clear description. The tool description does not add further semantic meaning beyond what the schema provides, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists every member in a workspace with specific fields (organization role, workspace role, status, timestamps) and explicitly distinguishes from sibling tools like get_workspace_member, update_workspace_member, and remove_workspace_member.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'Use this to find a user_id before get_workspace_member, update_workspace_member, or remove_workspace_member.' This tells the agent exactly when to use this tool and suggests alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only indicate non-read-only and non-destructive. The description adds crucial behavioral context: immediate routing of all callers and no rollback possibility. This is essential for understanding the irreversible impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: three sentences that front-load the core action, contrast with siblings, explain impact, and give a recommended workflow. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains the effect, irreversibility, and workflow. It complements the existing schema and annotations well. Could optionally mention error conditions or prerequisites, but given the output schema exists and annotations are adequate, it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameter descriptions. The description does not add further detail about the parameters themselves but provides contextual usage advice ('use list_prompt_versions to pick the version'). Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Publish a specific version of a prompt as the active default'. It also distinguishes itself from sibling tools 'promote_prompt' and 'update_prompt', making the action unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit when-to-use guidance by contrasting with 'promote_prompt' and 'update_prompt'. It also advises a workflow: use 'list_prompt_versions' to pick the version and 'update_prompt' if new content is needed before publishing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (destructiveHint=true), the description reveals that the new secret is exposed in the transcript, the previous secret remains valid during a transition period, and both secrets must never be logged. This adds critical behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences: purpose, behavioral detail, and actionable guidance. No fluff; front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers essential aspects (purpose, transition window, security). Lacks explicit error conditions or prerequisites, but given the output schema and full parameter descriptions, it is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds context about the transition period's effect (key_transition_expires_at) but does not significantly enhance parameter meaning beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Rotate an API key without changing its identity or scopes,' specifying the action and resource. It implicitly distinguishes from sibling tools like update_api_key by emphasizing identity and scope preservation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides post-rotation guidance (store securely, update callers, never log) and hints at the transition window, but does not explicitly state when to use rotate versus alternatives like create or update.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that the tool makes a billable model call, adding context beyond annotations. Annotations already indicate readOnlyHint=false and destructiveHint=false. The description adds value but could mention other side effects like logging.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first defines purpose, second provides usage guidelines. No wasted words; front-loaded with essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With output schema present and extensive parameter descriptions in schema, the description is largely complete. It covers purpose and billing context but could mention error handling or rate limits for a more complete picture.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds minimal parameter info beyond the schema, mostly restating that billing metadata is required. No significant enhancement.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description states 'Execute a prompt against the configured model and return the completion.' This provides a specific verb and resource, and distinguishes from siblings by mentioning render_prompt and validate_completion_metadata.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises to use render_prompt for template checking and validate_completion_metadata when billing fields are uncertain, given the billable nature of the call. Provides clear usage context and alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=false and destructiveHint=false but no further details. The description adds key behavioral traits: changes take effect immediately, type/sub-type stay fixed, and the call returns success without rotating the secret. This goes beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loaded with purpose and sibling differentiation, followed by behavioral details. Every sentence adds unique value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 11 parameters and existence of an output schema, the description is fairly complete. It covers key effects, constraints, and return behavior. However, it could mention that the 'id' is required and that other parameters are optional, but the schema already indicates this. No major gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and each parameter has a description. The description only lists general categories (name, description, scopes, defaults, limits) without adding detail beyond what's in the schema. Baseline 3 is appropriate since schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update an API key's name, description, scopes, defaults, or limits', specifying the verb and resource. It also explicitly distinguishes from sibling tools delete_api_key and create_api_key, which is essential for correct selection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description tells when to use this tool vs alternatives (unlike delete/create), but does not mention scenarios where it should not be used (e.g., changing type/sub-type is impossible). The immediate effect and fixed fields provide good context, but some exclusion guidance is missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds context about version creation and dry_run validation beyond the annotations. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with essential information, no redundant text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 14 parameters and existing output schema, the description covers core behavior, versioning, and validation with dry_run. It is complete enough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds value by clarifying partial updates and the preferred messages parameter for chat prompts.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates an existing prompt and creates a new archived version, distinguishing it from create_prompt and publish_prompt.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains partial updates, the role of publish_prompt, and provides guidance for multi-message prompts. It lacks explicit when-not-to-use scenarios but is generally clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds value beyond annotations by detailing that auth_config is merged, allowed_workspaces replaces mappings, and credential values are exposed to transcript. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three efficient sentences: purpose, key behavioral details, security note. Front-loaded with action, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers critical merge/replace behaviors and credential exposure. With output schema present, return value explanation is unnecessary. Minor gap: no prerequisites or error conditions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, baseline 3. Description adds meaning by specifying merge behavior for auth_config and replacement logic for allowed_workspaces, enhancing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Update at least one selected field on a Secret Reference', specifying verb and resource. Differentiates from sibling create/delete/get/list by emphasizing partial update.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description explains merging vs replacing behavior for auth_config and allowed_workspaces, providing clear context for use. Lacks explicit when-not to use, but siblings are distinct.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, idempotentHint=false, destructiveHint=false, and openWorldHint=true. The description adds the key behavioral constraint that only the role field changes and recommends a confirmation step before updating, which is useful context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two focused sentences: the first states the action and required keys, the second states the scope and a pre-check workflow. There is no redundant or filler content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the schema covers all parameters, annotations cover the safety profile, and an output schema exists, the description is sufficient for selection and invocation. It includes the purpose, the specific keys, the role enum context, and a confirmation step, though it could briefly hint at alternative member-management tools for non-role updates.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers all three parameters thoroughly, including the role enum and user_id UUID constraints. The description reinforces that role is the only mutable field, adding semantic clarity beyond the schema fields.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb ('Update') and names the exact resource ('workspace member's role') plus the identifying keys ('by workspace_id and user_id'). It distinguishes itself from sibling member tools by explicitly limiting the scope to role changes only.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly states the tool is for role updates only and tells the agent to use list_workspace_members or get_workspace_member to confirm the current assignment first. It does not explicitly name alternative mutation tools (e.g., remove_workspace_member), but the scope exclusion ('Only the role changes here') provides clear when-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses a key behavioral trait—that creation does not start processing—and the expected follow-up workflow. Also notes the 403 error on non-Enterprise plans. While annotations indicate readOnlyHint=false, the description adds meaningful context about the asynchronous workflow and access restriction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each with a distinct purpose: action, workflow steps, and access caveat. Information is front-loaded and no redundant wording appears.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a moderately complex tool with 10 parameters and an output schema, the description covers the critical workflow (create→start→retrieve) and enterprise restriction. Return values are already documented by the output schema, so no additional explanation is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides 100% coverage with detailed descriptions for all 10 parameters, including enums for requested_fields. The description only adds generic 'filters and requested fields', so it does not need to elaborate further. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a log export definition with filters and requested fields' with a specific verb and resource. It further distinguishes the tool from siblings by explicitly noting that it 'only sets up the export and does not start processing', preventing confusion with start_log_export or download_log_export.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit workflow guidance: 'call start_log_export next, then use get_log_export or download_log_export'. Also mentions Enterprise-gating and 403 behavior, which helps decide when to use this tool versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate it is not read-only (write operation), not destructive, and open world. The description adds that it returns the new label id and does not assign to any versions. This provides useful behavioral context beyond annotations, but could mention if there are any side effects on existing labels or other entities.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the purpose, and every word adds value. No redundant or vague phrasing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists, the description does not need to detail return structure but mentions the id. It covers scope requirement, lack of assignment, and optional color. For a creation tool with 5 parameters, this is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all parameters with descriptions. The description adds that either organisation_id or workspace_id is required (though schema marks only name as required), which is a critical constraint. It also clarifies that color_code is optional and defaults to a default color. This adds meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a prompt label' and gives concrete examples like 'production, staging, or experiment'. It distinguishes the tool from siblings by noting that it does not assign the label to any versions yet, which sets it apart from assignment or update actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies that either organisation_id or workspace_id is required to set scope, providing clear guidance on parameter usage. It also notes that the label is not assigned to versions, implying a subsequent step. However, it does not explicitly state when to use this tool versus alternatives like update_prompt_label.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the destructiveHint annotation, the description adds key details: permanent deletion, removal of all versions, and breaking anything pointing to the slug. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence with two concise clauses, front-loaded with the action. Every word adds value; no unnecessary text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with an output schema (exists elsewhere), the description covers the action, permanence, and a crucial precaution. It is fully adequate for an agent to use without ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides 100% coverage for the single parameter 'slug' with description 'Configuration slug to delete'. The description does not add additional parameter details, meeting the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Delete a config by slug.' It explicitly names the resource and action, and the sibling tools (create_config, update_config, get_config, list_configs) make it distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description warns of permanence and breaks dependencies, and suggests checking list_config_versions first. It provides clear when-to-use/not-use context, but lacks explicit mention of alternatives like update_config for deprecation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark destructiveHint=true and readOnlyHint=false. The description adds specific behavioral context: 'This cannot be undone, immediately breaks callers using the slug,' which goes beyond annotations. However, it does not detail any rate limits or permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no unnecessary words. It front-loads the core action ('Delete a prompt...'), then adds critical caveats. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive one-parameter tool with an output schema, the description covers what is deleted, irreversibility, impact on callers, and prerequisite checks. It is complete given the tool's simplicity and the presence of annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage for the single parameter 'prompt_id' with a clear description ('Prompt ID or slug to delete'). The tool description adds no additional meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Delete a prompt and all its versions by id.' It uses a specific verb ('Delete') and resource ('prompt and all its versions'), distinguishing it from sibling tools like 'delete_prompt_label' or 'delete_prompt_partial'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'should only be used after checking list_prompt_versions or confirming you do not need an audit trail.' It warns about irreversibility and impact on callers ('immediately breaks callers using the slug'), helping decide when to use this tool versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes beyond annotations by detailing the irreversible nature and downstream effects on versions and workflows. Annotations only flag destructiveHint=true, but the description explains what gets destroyed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states the action, second explains key consequences. Every word is necessary, no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has one parameter and an output schema (though not shown, it exists). The description covers the essential behavioral impact, making it complete for this simple tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema coverage and only one parameter, the schema already describes the parameter as 'Label ID to delete.' The description does not add additional semantics, so baseline score applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Delete a prompt label by ID.' It uses a specific verb and resource, distinguishing it from sibling tools like update_prompt_label or create_prompt_label.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit consequences: 'cannot be undone', 'versions carrying the label lose it', and 'any workflow resolving by that label will need a replacement.' This helps the agent decide when to use it cautiously, though it does not explicitly name alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations include destructiveHint=true, so the safety profile is already indicated. The description adds that the invite link is revoked and that existing users are unaffected, providing extra context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no redundant information. Every phrase adds value, and the structure is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter and an existing output schema, the description covers purpose, scope, and exclusions thoroughly, leaving no ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with one parameter (invite_id) already documented. The description does not elaborate on parameter meaning beyond schema, but the schema itself is clear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Delete a pending invite and revoke its invite link,' specifying the action and resource. Distinguishes from 'delete_user' by noting it does not affect existing users, helping the agent differentiate among sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly provides when to use (delete pending invite) and when not to (does not affect existing users), and directs to 'delete_user' for full removal, offering a clear alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, but the description adds context beyond these: the export must be completed, and non-Enterprise plans receive a 403. This is valuable behavioral information not present in annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the main purpose, and followed by necessary conditions and alternative tool guidance. Every sentence contributes value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool with an output schema, the description covers prerequisites, alternative tools, error conditions (403), and access restrictions, making it fully complete for selecting and invoking the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters with a clear description for export_id. The tool description adds no additional parameter-level semantics, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Get a signed URL for downloading a completed log export.' It distinguishes from sibling tools by specifying the export must be completed and referencing related tools like get_log_export and start_log_export.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance on when to use: the export must already be finished; directs the user to get_log_export to confirm readiness and start_log_export if it has not run yet. Also notes the Enterprise-gating prerequisite and potential 403 error.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, idempotent, non-destructive behavior. The description adds that it returns 403 on non-Enterprise plans and that it provides time-series data, which is useful behavioral context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences covering purpose, output fields, alternative use case, and enterprise restriction. No wasted words, front-loaded with key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema, annotations, and sibling tools, the description is complete: explains what it returns, when to use it, and the enterprise limitation. No critical gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 29 parameters. The description does not add new meaning beyond summarizing the output fields; it does not augment parameter understanding, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves cache-hit-only latency time-series data with specific fields (summary.total_latency, etc.) and explicitly contrasts with get_latency_analytics for all requests. This distinguishes it well from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use (evaluate cached-response speed) and when not (use get_latency_analytics for all requests). Also mentions Enterprise-gating and 403 response for non-Enterprise plans, providing clear usage context and alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, covering the safety profile. The description adds value by specifying the retrieval method (by id or slug) and the returned fields, which are not in the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loads the verb and resource, and every sentence serves a clear purpose. No unnecessary words or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, safe read operation, existing output schema), the description provides all necessary context: what it does, how to use it, and when to prefer it over siblings.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with a description for 'collection_id' that already mentions 'Collection ID or slug'. The description does not add new semantic meaning beyond the schema, so baseline is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Fetch', the resource 'one collection by id or slug', and specifies the return fields. It distinguishes from the sibling tool 'list_collections' by mentioning the browsing vs. targeted retrieval use case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises when to use this tool ('when you already know the target') versus the alternative 'list_collections' ('when browsing'), providing clear guidance on selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds that the tool is enterprise-gated and returns 403 for non-Enterprise plans, which is valuable behavioral context beyond annotations. However, it does not mention rate limits, pagination, or other potential behaviors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences that front-load the core purpose and immediately provide usage guidance. Every sentence adds value with no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema (covering return values), 29 parameters (2 required), and annotations, the description is complete enough. It explains the tool's role, when to use it, and Access restrictions. No additional information seems necessary for an analytics retrieval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description does not add parameter-specific meaning beyond what the schema already provides; it simply states the overall purpose. No additional guidance on how to use parameters is given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get feedback time-series data grouped by model, with per-model counts over time.' It uses a specific verb ('Get') and resource ('feedback time-series data grouped by model'), and distinguishes from the sibling tool 'get_feedback_analytics' by noting that it provides per-model breakdowns.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use this to compare feedback volume and satisfaction across models; use get_feedback_analytics for the overall total instead.' It also mentions that the tool is 'Enterprise-gated' and 'Returns 403 on non-Enterprise Portkey plans,' providing clear when-to-use and alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior. The description adds useful context beyond annotations: it discloses the return payload fields and the Enterprise plan requirement that may cause a 403. This gives the agent important behavioral and error-handling context without contradicting the structured hints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, each earning its place: function and return value, usage distinction, and access restriction. It is front-loaded with the primary action and avoids any filler. This is a model of concise, high-signal writing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one required parameter, a well-populated schema, an output schema, and rich annotations, this description provides all necessary context: what it retrieves, when to use it, and what plan is required. There are no significant gaps given the tool's low complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema fully covers the single parameter export_id with a description ('The unique ID of the log export'). The description adds no additional parameter-level syntax or format detail, but because schema coverage is 100%, the baseline of 3 is appropriate; the tool's simple structure does not require further elaboration.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('Fetch one log export job by export_id') and the resource (a single log export job), and lists what it returns (status, filters, requested fields, file metadata). This distinguishes it from the sibling tool list_log_exports, which is explicitly named as an alternative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit when-to-use guidance: 'Use this when you already know the target; use list_log_exports for a workspace-wide overview.' This clearly separates it from the alternative and also notes the Enterprise-gated access and 403 behavior, which helps set expectations for use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true, destructiveHint=false, idempotentHint=true, so the description does not need to repeat these. It adds value by specifying the return fields (parent integration, status, created time), which gives behavioral insight beyond the schema. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two sentences, each earning its place. The first sentence states purpose and return info, the second provides usage guidance. No extraneous words, front-loaded effectively.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (single parameter, output schema exists), the description sufficiently covers what the tool does, what it returns, and when to use it. Annotations and output schema cover the rest, so no gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (the parameter 'id' has a clear description). The description adds only marginal value by mentioning 'by id or slug', which is already in the parameter description. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verb 'Retrieve' and resource 'one MCP server by id or slug', clearly stating the action and input. It distinguishes itself from get_mcp_integration by specifying when to use this tool for server records rather than integration config.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use this tool vs. the alternative get_mcp_integration: 'use get_mcp_server when you need the server record rather than the integration config.' This provides clear context and an alternative, meeting the when/alternatives criterion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, so description needs little behavioral context. It adds value by stating what fields are returned (template, parameters, model config), which is helpful but not critical.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no fluff. Every word earns its place, front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that output schema exists, the description appropriately highlights key return fields. It covers purpose, prerequisite steps, and output summary, making it complete for a retrieval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Both parameters are fully described in the schema (100% coverage). The description adds no additional meaning to the parameters beyond usage guidance, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it retrieves a specific prompt version by UUID, distinguishing it from list_prompt_versions and get_prompt. The verb 'Retrieve' and resource 'prompt version' are specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs to use list_prompt_versions first to find the ID, providing clear when-to-use guidance and a direct reference to a sibling tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already define safety profile (readOnly, idempotent, non-destructive). Description adds detail about return content (conditions and grouping), which is useful but doesn't introduce new behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states purpose and return, second gives usage guidance. No redundant words, efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With output schema present, description doesn't need to detail return values. It covers purpose, return content, and usage context fully.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema describes 'id' parameter fully (100% coverage). Description only repeats 'by id', adding no extra semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Get', resource 'rate limit', and scope 'by id' while specifying return content (full conditions and grouping). Mention of sibling tool list_rate_limits provides differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises to use list_rate_limits first to discover ids, giving clear when-to-use guidance and an alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, destructiveHint false, etc. The description adds context that the tool only returns 'accepted' users and is Enterprise-gated, providing useful behavioral detail beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core purpose, no redundant information. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get-by-id tool with one parameter and an output schema, the description covers the return fields, how to find the id, alternatives for different states, and access restrictions. Complete for its complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The sole parameter user_id is described in schema as 'The user ID to retrieve'. The description adds no new semantic meaning to the parameter itself, but does provide context on obtaining the id from list_all_users, which is more about usage than parameter semantics. Schema coverage is 100%, so baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'one accepted user', and distinguishes from siblings like list_all_users and get_user_invite by specifying the scope and alternative uses.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly guides when to use list_all_users (to find id from name/email) and get_user_invite (for pending invitations), and warns about Enterprise gating and 403 response.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds that it returns specific fields (email, role, status, expiry) and that it's for pending invites. There is no contradiction, and the behavior is well disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, no unnecessary words. It front-loads the purpose and includes usage guidance in a compact form.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a single parameter, strong annotations, and the presence of an output schema (context shows has output schema: true), the description is sufficient. It covers what the tool does, what it returns, and when to use it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter is invite_id, which has a clear schema description ('The invite ID to retrieve'). The tool description does not add further semantic detail beyond what the schema already provides. With 100% schema coverage, the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a single invitation by ID and lists the returned fields (email, role, status, expiry). It also distinguishes from get_user by explicitly stating 'Use this for pending invites only; use get_user for accepted users.'
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'Use this for pending invites only; use get_user for accepted users.' This tells the agent when to use this tool vs. an alternative sibling.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, destructiveHint, idempotentHint, openWorldHint. Description adds that it returns 'full details, including defaults and the complete member list', providing useful context beyond the annotations. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states purpose and what it returns, second gives usage guidance. Front-loaded and efficient with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists, and description adequately states what is returned (full details including defaults and member list). Sufficient for a simple retrieval tool with one parameter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 1 parameter with 100% description coverage, so baseline is 3. Description does not add additional parameter semantics beyond what schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Get one workspace by id' with specific verb and resource, and distinguishes from sibling tool list_workspaces by mentioning 'return its full details, including defaults and the complete member list'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Contains explicit guidance: 'Use this when you need membership detail; use list_workspaces for an overview.' Clearly tells when to use this tool vs alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, so the basic safety is covered. The description adds value by detailing the kind of data returned (scopes, limits, expiration), which helps set expectations beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, and the second sentence provides differentiation and return summary without any wasted words. Every sentence serves a clear function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of a full output schema and annotations covering safety, the description succinctly covers the tool's purpose, differentiation, and the key aspects of its return data. No gaps remain for a competent agent to use this tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with all parameters documented. The description does not add new information about the parameters themselves, but it mentions the return data types (e.g., workspace/user scope) that indirectly relate to filtering via workspace_id. This meets the baseline expectation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists Portkey API keys for auditing, specifying the exact resource and purpose. It also distinguishes itself from the sibling tool list_virtual_keys, which is for provider keys.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use this tool ('for API keys only') and provides a clear alternative ('use list_virtual_keys for provider keys'), giving the agent a direct comparison.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description adds that it returns a summary view with specific fields, which is useful context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first presents purpose and output, second provides usage guidance. No wasted words, front-loaded, and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given annotations, schema, and output schema, the description is complete. It covers the tool's functionality and workflow context without missing essential information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline 3 is appropriate. The description does not add meaning beyond the schema for the two optional pagination parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists configs with specific fields (id, slug, name, status, workspace, timestamps). It also distinguishes from get_config by noting that this is a summary view.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance: use this to find a slug, then use get_config for full settings before updating/deleting. This clearly indicates when to use this tool vs alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover read-only/idempotent/destructive hints. The description adds valuable context about required API key scope, Enterprise-gated availability, and the 403 behavior on non-Enterprise plans, which goes beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, all directly informative, with front-loaded primary action. No redundant or filler content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter read-only list tool with an output schema, the description covers purpose, usage, auth, enterprise gating, and returns meaningful constraints. It is fully sufficient for an agent to select and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single parameter, so the schema fully documents 'organisation_id'. The description adds no additional parameter-level detail, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists workspaces excluded from organisation-wide input guardrails for one organisation, with a specific verb and resource. It also distinguishes itself by noting it only reads input exclusions, not the default guardrail list, which differentiates it from sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: to audit exceptions or establish current state before the matching update tool. It also clarifies what it does not do (does not return default guardrail list), providing clear context versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, openWorldHint, idempotentHint, destructiveHint false. Description adds that it returns total plus specific fields (ids, names, enabled state, usage limits, rate limits), and enterprise restriction with 403. No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences covering purpose, use case, and return value plus constraints. No fluff, front-loaded with main action. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given output schema exists, description covers return fields, enterprise gating, and error condition. Adequate for a paginated list tool with read-only behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 3 parameters are described in the schema (slug, page_size, current_page) with 100% coverage. Description adds no extra semantic meaning beyond the schema. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'List workspaces that can use an integration, with their limits.' Verb+resource is specific. Differentiates from siblings like list_mcp_integration_workspaces by focusing on integrations (not MCP). Also adds audit use case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use ('audit access or confirm per-workspace cost and rate settings'). Also notes Enterprise-gating and 403 error. Does not explicitly exclude alternatives, but clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the tool as read-only (readOnlyHint: true), non-destructive, idempotent, and open-world. The description adds value by specifying that it returns 'total plus enabled-state entries' and positions it as a preparatory step for toggling. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, front-loaded with the primary purpose, and no redundant information. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple listing tool with one parameter and an assumed output schema, the description provides the purpose, return summary, and usage context (preparation for update). It is complete enough for an agent to understand when and how to use it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter 'id', and its description is adequate. The tool description does not add extra parameter details, but since the schema already covers it, a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List', the resource 'capabilities exposed by the external MCP server for an integration', and distinguishes it from siblings like 'update_mcp_integration_capabilities' and 'list_mcp_integrations'. It also specifies the return content (total plus enabled-state).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises to 'use before update_mcp_integration_capabilities when you need to compare the current surface', providing clear context and an alternative. This helps the agent decide when to invoke this tool versus the update sibling.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false. Description adds value by disclosing pagination and total count behavior, enhancing transparency beyond structured fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with front-loaded purpose ('List MCP servers in the organization'), no redundant or extraneous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given annotations and output schema coverage, the description fully covers key aspects: purpose, pagination, sibling differentiation, and usage context, leaving minimal gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline 3. Description does not add parameter-specific details beyond schema, but implies pagination and filtering context, maintaining adequacy.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'List MCP servers in the organization' with a specific verb and resource. It explicitly distinguishes from siblings by directing to get_mcp_server for details and list_mcp_integrations for parent integration, ensuring no confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: 'Returns paginated server records plus total for discovering server IDs' and contrasts with alternatives for one server or parent integration, covering when and when not to use the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, non-destructive, idempotent, and openWorld. Description adds that the tool returns specific fields (version numbers, descriptions, status, timestamps), giving extra clarity on output content beyond the output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states purpose and output, second specifies usage scenario. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With one simple parameter, full schema coverage, output schema present, annotations covering safety, and a clear link to a sibling tool (publish_partial), the description is complete and sufficient for an agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% coverage with clear parameter description. Description does not add significant extra meaning beyond the schema, but the mention of 'version_id' in context provides a slight connection.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it lists all versions for one partial, including specific fields (version numbers, descriptions, status, timestamps). It implicitly differentiates from sibling tools like list_prompt_versions by focusing on partials.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use this when you need history or want to choose a version_id before publish_partial,' providing clear when-to-use context and linking to a related action.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the behavioral profile is clear. The description adds value by listing the return fields (total, provider name, slug, integration, status, limits, expiration, reset flags), which provides context beyond annotations without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no fluff: first sentence states the action and scope, second gives usage guidance and return field enumeration. Information is front-loaded and every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (3 parameters, annotations present, output schema exists), the description covers purpose, usage guidance, and return fields completely. Pagination details are already in the schema, so no further elaboration needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with each parameter described (page_size, current_page, workspace_id). The description does not add additional parameter-specific guidance beyond what the schema provides, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'List workspace-scoped provider instances and their limits or status,' which is a specific verb+resource+scope. It also distinguishes from the sibling tool list_integrations by noting the different scopes (workspace vs org-level).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool ('to find provider slugs for workspace-level updates') and when to use an alternative ('use list_integrations for the org-level source connection'). This provides clear context and exclusion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds value by listing returned fields and suggesting it's a discovery tool, though it could mention pagination or ordering if applicable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, first lists fields, second provides usage guidance. No wasted words, front-loaded with key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given one optional parameter, existing output schema, and thorough annotations, the description is complete enough: it explains what is listed and when to use it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers the single parameter workspace_id at 100%. Description does not add extra meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists rate limits with specific fields (id, type, unit, value, status, scope, conditions, grouping), differentiating it from sibling tools like get_rate_limit (single) or update_rate_limit (mutate).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises to use this tool before get_rate_limit, update_rate_limit, or delete_rate_limit, providing a clear workflow and context for when to choose this over alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite annotations already declaring readOnlyHint, openWorldHint, idempotentHint, and destructiveHint, the description adds valuable context: the tool is Enterprise-gated and returns 403 on non-Enterprise plans. It also clarifies scope limitations (directory groups only, no mappings/members), which go beyond annotation data. No contradiction exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences: purpose, usage guidance, and behavioral/entitlement notes. Every sentence adds distinct value, and the most important information (what it does) is front-loaded. No filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, return values need no explanation. The description covers purpose, usage context, scope limitations, and error behavior for a 3-parameter read-only tool, making it fully complete for an agent to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with each parameter (page, search, page_size) already having meaningful descriptions. The tool description does not add additional parameter semantics, but the schema carries the burden effectively, meeting the baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Search and page through identity-provider groups synchronized to Portkey over SCIM.' It clearly distinguishes itself from sibling tools by noting it 'reads directory groups only and does not show their workspace mappings or individual members,' which separates it from list_scim_workspace_mappings and related tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs when to use it: 'Use this to resolve a group ID or exact display name before create_scim_workspace_mapping.' It also states what it does not do (show workspace mappings or members), offering clear exclusionary guidance. However, it does not explicitly name an alternative tool for those excluded cases, so it falls just short of full alternative specification.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, idempotent, non-destructive behavior. The description adds layering context (returns 403 on non-Enterprise plans) and scope (only pending/sent invites).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences. First sentence states purpose and output fields, second gives usage guidance, third notes access restriction. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, return value documentation is unnecessary. The description covers purpose, usage context, and access restriction adequately for a list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for both parameters (page_size, current_page). The description adds no additional parameter details beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists pending and sent invitations with specific fields. It differentiates from the sibling tool 'list_all_users' by explicitly stating to use that for accepted users.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: 'Use this to check invite state; use list_all_users for users who already accepted.' Also notes enterprise gating and 403 response.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that it immediately changes the default for all prompts, replaces the active version, and has no rollback path. This adds value beyond annotations (which indicate non-read-only behavior).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states purpose and differentiation, second gives usage guidance and consequences. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given simple two-param schema, annotations, and presence of an output schema, the description covers purpose, usage, effect, and caution. Sufficient for correct tool invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters with descriptions. The description provides usage context (e.g., 'use after list_partial_versions') but adds no new meaning to the parameters themselves.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool publishes a specific partial version as default and distinguishes it from update_prompt_partial, which creates a draft. It specifies the effect on partial resolution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises to use after list_partial_versions to pick a version_id, describes immediate changes and no rollback path. Lacks explicit 'when not to use', but context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true and readOnlyHint=false. The description adds valuable context by clarifying that workspace access is revoked but the user remains in the organization, which helps the agent understand the exact scope of destruction. It does not cover reversibility or permissions, but the essential behavioral traits are disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the primary action, and contains no fluff. The second sentence adds important clarification about what the tool does not do, which is essential for disambiguation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete for the tool's complexity: there is an output schema, the parameters are fully described in the schema, annotations cover safety traits, and the description provides the needed scope and alternative guidance. No critical information is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both required parameters (user_id and workspace_id) already documented in the schema. The description does not add further parameter-level detail, but the schema carries the full semantic burden, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action: 'Remove a user from a workspace and revoke workspace access.' It clearly distinguishes from the sibling delete_user by noting this action does not delete the user from the organization, making the tool's purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly provides usage guidance: use this tool to remove a user from a workspace, and use delete_user for full removal from the organization. This directly addresses when to use this tool versus an alternative, which is strong guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and non-destructive. Description adds important behavioral context: it substitutes variables, returns final messages, and does not call the model. This clarifies the exact operation beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first explains what the tool does, second provides usage guidance and distinguishes from a sibling. No unnecessary words, front-loaded with essential info.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with nested parameters and output schema, the description covers the core purpose, usage context, and comparison to sibling. It notes return of final messages, which suffices given output schema exists. Complete for informed selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description does not repeat parameter schemas but implicitly connects the 'variables' parameter to substitution. No extra detail beyond schema, but given full coverage, this is adequate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it renders a prompt by substituting variables and returning final messages without calling the model. It explicitly distinguishes from the sibling tool run_prompt_completion, which invokes the model. The verb and resource are specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use this tool: 'Use this to verify template output before a completion'. Names the alternative tool run_prompt_completion for when model invocation is needed. Provides clear context that this is for verification only.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate non-readonly, non-destructive, non-idempotent, open-world. The description adds that only value, weight, and metadata are changed, trace linkage is immutable, and it returns updated status and IDs. This provides useful context beyond annotations, though it could mention potential side effects (e.g., impact on analytics).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the action and return, then constraints and alternative. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (4 parameters, output schema exists) and the presence of sibling alternatives, the description covers the essential aspects: purpose, modifiable fields, immutability of trace linkage, and usage guidance. No critical information is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for all four parameters. The description summarizes which parameters are modifiable but does not add new semantic details beyond the schema. The mention of return values is about output, not input parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update an existing feedback record by ID' with a specific verb and resource. It details what fields are changed (value, weight, metadata) and distinguishes from the sibling tool create_feedback, which is for new records.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool (to update existing feedback) and when not (use create_feedback for a new record). It also clarifies that trace linkage is immutable, guiding parameter selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and idempotentHint=true. The description adds valuable context beyond annotations: 'These changes affect every workspace using the integration' and 'Returns success and the number of models updated.' This improves transparency, though it doesn't cover all nuances like atomicity or rollback behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is four sentences, each earning its place: first lists actions, second clarifies a parameter, third warns about scope and gives guidance, fourth notes the return. It is front-loaded and concise without waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with 3 parameters and a nested models array, the description covers purpose, safety, scope, sequencing, alternatives, and return value. An output schema exists, so detailed return fields do not need to be explained. The description is complete and well-rounded.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description summarizes parameter effects (e.g., 'allow_all_models controls whether future provider models start enabled') but does not add significant new meaning beyond the schema definitions. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb and resource: 'Bulk enable or disable integration models, register custom or fine-tuned models, set per-model hosts and headers, and attach static token pricing.' It enumerates distinct actions, which distinguishes it from sibling tools like list_integration_models or delete_integration_model.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance: 'inspect list_integration_models first' and 'use get_model_pricing when deriving custom rates.' It names alternative tools and gives sequencing context, fulfilling the when-to-use vs alternatives criterion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations show readOnlyHint=false, destructiveHint=false, idempotentHint=true. The description adds meaningful context: reversible toggle, immediate exposure, and that only named capabilities are affected. No contradictions. It does not cover rate limits or auth, but annotations already indicate non-destructive and idempotent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core action, then key behavioral points. Every sentence is purposeful and concise. No fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (bulk update with multiple capability types), the description is self-sufficient. It explains purpose, usage, side effects, and prerequisite data sources. The output schema existence is not critical as it mentions a confirmation message.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions. The description adds value by explaining the effect of the array (state toggle), reversibility, and the immediate visibility. This enriches the agent's understanding beyond raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with 'Bulk enable or disable capabilities (tools, prompts, resources) on an MCP integration,' clearly specifying the verb (enable/disable), resource (capabilities on an MCP integration), and distinguishing it from sibling tools like update_mcp_integration and list_mcp_integration_capabilities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides explicit context: reversible toggle, not deletion; only named capabilities change state; immediate for connected users; re-running with enabled flipped restores them. It also instructs to source the integration id from list_mcp_integrations and current states from list_mcp_integration_capabilities. No direct when-not-to-use, but implied through reversibility.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description notes that changes apply immediately, adding behavioral context beyond annotations. No contradictions with annotations (readOnlyHint=false, destructiveHint=false). Adequate for a simple update tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states purpose, second provides usage guidance. No redundant information, front-loaded with key action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, an output schema exists, and the description covers what fields are updatable, immediate effect, and sibling tool delegation. Complete for this context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for each parameter. The description adds no additional meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Update') and the resource ('MCP server') and specifies the fields affected (name or description). It distinguishes from the sibling tool 'update_mcp_integration' by noting that URL and auth are handled separately.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly tells the agent when to use this tool versus the sibling ('use update_mcp_integration for those fields'), providing clear guidance on tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate mutation (readOnlyHint=false) but no destruction. The description adds that the tool only affects label assignment and clarifies null behavior ('null clears the label'). This adds meaningful context beyond the basic annotation flags.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences. The first states the core purpose, the second elaborates on null behavior and prerequisite workflow. No unnecessary words; all content earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with three required parameters and an output schema, the description covers the essential: what it does, how null works, and where to get label IDs. No gaps for this complexity level.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, providing baseline 3. The description adds value by explaining the label_id parameter's role in assigning/removing labels and guiding the agent to use 'list_prompt_labels' for ID lookup. This goes beyond schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it updates a prompt version's label assignment, with specific verbs ('assigns', 'removes', 'clears'). It distinguishes itself from siblings like 'update_prompt' (which updates the prompt itself) and 'list_prompt_labels' (for looking up label IDs).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context: it only assigns or removes a label, and recommends looking up IDs with 'list_prompt_labels'. It does not explicitly state when not to use it or list alternative tools, but the scope is well-defined.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate mutation (readOnlyHint=false) and non-destructive nature. Description adds key behavioral context: immutability of conditions/group_by and the need to fetch the full policy via get_rate_limit. This goes beyond annotations but could mention if updates trigger immediate effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no fluff. Important information is front-loaded and every sentence contributes value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (partial update with immutable fields) and the presence of an output schema, the description provides sufficient context: what can be updated, what is immutable, and a suggested prerequisite step.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% description coverage for all 4 parameters. The description adds minimal additional meaning beyond 'by id' and which fields are updatable, so a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it updates a rate limit's name, unit, or value by id. Distinguishes from siblings by noting that conditions and group_by are immutable and referencing get_rate_limit for the full policy.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly mentions when to use get_rate_limit first if the full policy is needed, and clarifies that conditions and group_by cannot be updated, guiding the agent to alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds behavioral context beyond annotations: 'New values apply immediately to tracked usage', 'conditions and group_by are immutable after creation', and 'returns the updated id without clearing accumulated usage'. It aligns with annotations (readOnlyHint=false, destructiveHint=false) and provides useful caveats.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that packs purpose, sibling distinction, usage notes, and behavioral caveats without redundancy. It is efficient, though breaking into multiple sentences could improve readability slightly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's purpose, parameters, sibling differentiation, and key behavioral traits (immediate application, immutability, no usage reset). With an output schema present and good annotations, it provides a solid understanding for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
While the schema has 100% coverage on parameter descriptions, the description adds value by grouping the updatable fields and noting immutability of certain fields. This helps the agent understand which parameters can be modified and which cannot, enhancing the schema info.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and the specific resource 'usage limit' along with the fields that can be updated. It also contrasts with the sibling tool 'update_rate_limit' by noting it tunes request throttling, making the purpose distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: it says to update 'by id' and contrasts with 'update_rate_limit'. It also notes that 'conditions and group_by are immutable after creation' and directs to 'reset_usage_limit_entity' for resetting usage, indicating when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate a mutation tool (readOnlyHint=false). Description adds precondition that user must exist in org, but does not disclose potential error cases like duplicate membership or permission requirements. Good but could be more thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: purpose and usage guidance. No redundant information, well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given moderate complexity and presence of output schema, description covers key preconditions and usage steps. Could mention error handling for duplicate members, but overall sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all parameters with descriptions. Description adds value by clarifying user_id must be UUID (not email) and referencing list_all_users, which goes beyond schema details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Add an existing org user to a workspace with a role,' specifying verb, resource, and constraints. It distinguishes from sibling tools like invite_user and list_all_users by referencing them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly provides prerequisites: requires UUID user_id, use list_all_users to find it, and invite_user if not in org. This gives clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds critical behavioral details beyond annotations: warns that the key secret is returned only once and visible in transcripts, and states that the key grants immediate access. No contradiction with annotations (readOnlyHint=false, etc.).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences with clear structure: purpose, type distinction, security warning, activation behavior, and requirements. Front-loaded with most important info.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, security, activation, and key requirements. With output schema present and 14 parameters, the description is adequate but could mention optional fields like rate limits or defaults.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for each parameter. The description reinforces key constraints (workspace_id required for workspace type, user_id for user sub-type) and scopes, adding marginal value over schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Explicitly states 'Create a Portkey API key for auth' and distinguishes between org and workspace keys. Clearly differentiates from sibling tools like delete_api_key and update_api_key.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context on when to use each key type (org vs workspace) and prerequisites (workspace_id for workspace keys, user_id for user sub-type). However, does not contrast with alternatives like create_virtual_key.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds significant behavioral context beyond annotations: key is write-only, secret_mappings can resolve keys or configuration fields at runtime, and pricing_adjustments affect cost accounting. It also discloses that the integration becomes the source for workspace providers and returns the new integration id and slug. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise yet comprehensive, with each sentence serving a distinct purpose: purpose, key field explanations, behavior controls, security notes, and follow-up steps. It is front-loaded with the core action and avoids redundancy, making it effective for a tool with 19 parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex creation tool, the description covers essential aspects: purpose, key parameters, conditional behavior, follow-up actions, and return value. The output schema further defines the return. It lacks explicit prerequisites (e.g., existence of ai_provider_id) but overall provides sufficient context for an agent to invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
While the schema already covers all 19 parameters (100% coverage), the description adds valuable semantic grouping: ai_provider_id identifies the backend, provider-specific fields configure cloud providers, and create_default_provider controls automatic provider creation. It also explains the relationship between key and secret_mappings, going beyond individual parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's primary function: 'Create an AI-provider integration that becomes the source for workspace providers.' It distinguishes this from sibling tools like create_provider and create_mcp_integration by specifying the integration's role. The mention of provider-specific fields for Azure, Bedrock, Vertex, and custom hosts further clarifies its scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear guidance on when to use the tool, including context for workspace-scoped integrations via create_default_provider. It also directs users to use update_integration_models and update_integration_workspaces after creation. However, it does not explicitly state when not to use this tool or mention alternatives like create_provider for different scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true. The description goes further by specifying that deletion is irreversible and will break prompts, configs, and virtual keys, adding valuable behavioral details beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with action, then consequences and alternative. Every word is necessary and there is no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With destructiveHint=true and clear description of consequences, the tool is well-specified. However, it lacks details about permissions or rate limits, which would make it fully complete. Output schema existence is implied but not detailed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% description coverage for both parameters (slug, workspace_id). The description does not add extra meaning beyond what the schema already provides, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Delete' and the resource 'workspace provider by slug'. It also distinguishes itself from the sibling tool 'delete_integration' for the org source, eliminating ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use (delete a workspace provider) and when not to (use delete_integration for org source). Also warns about irreversibility and impacts on other entities, providing clear context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, non-destructive, idempotent behavior. The description adds that it returns full check and action configuration, status, and ownership, which is valuable beyond annotations. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states the purpose, second provides usage guidance. No unnecessary words, front-loaded with key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and an output schema (present but not shown), the description covers the purpose, usage context, and return content well. Slight deduction for not mentioning pagination or filtering, but not needed given the fetch-by-ID nature.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with a clear description of guardrail_id. The description mentions 'by id or slug', which matches but does not add significant new semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description specifies 'Fetch one guardrail' by id or slug, clearly defining the verb and resource. It distinguishes from sibling tool list_guardrails by mentioning to use that for discovery first.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states to use this tool before update_guardrail or delete_guardrail when the exact enforcement policy is needed, providing clear context and alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds critical behavioral context: Enterprise-gated and returns 403 on non-Enterprise plans, which annotations do not cover.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise: three sentences, front-loaded with what the tool does, then usage guidance, then behavioral note. Every sentence adds value, no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (29 parameters, legacy/structured aliases, output schema exists), the description covers output details, usage context, and access restrictions. Missing explicit mention that output is time-series, but implied. Slightly short of 5 due to lack of more detail on time-series structure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description does not add any parameter-level information beyond the schema; it only mentions output fields. Thus, no additional value for parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'get' and resource 'latency time-series data', lists output fields (summary.avg_latency_ms, etc.), and distinguishes from sibling 'get_cache_hit_latency' by stating when to use each.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: 'to spot slowdowns and regressions'; provides alternative: 'use get_cache_hit_latency when you only want cache-hit latency'; also notes Enterprise-gating, guiding non-Enterprise users away.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations (readOnlyHint=false, destructiveHint=false) indicate a write operation. The description adds that writes are immediate, fail on unmatching provider, and span fields enable trace hierarchy. All behaviors are disclosed without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences front-loaded with purpose and context. Every sentence provides meaningful information without redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool complexity (17 parameters) and existing schema/annotations/output schema, the description covers purpose, usage condition, failure case, and tracing guidance. It could optionally mention what the function returns, but output schema likely handles that, so it is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 17 parameters have descriptions in the schema (100% coverage), so baseline is 3. The description adds value by explaining span fields 'stitch trace hierarchies together' and the failure condition for request_provider, which goes beyond the schema's generic descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Insert log records for requests that bypassed the gateway' – a specific verb and resource with context. It distinguishes from sibling tools by focusing on bypassed requests, which is unique among the many create and log tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies when to use (bypassed gateway requests), includes a failure condition ('call will fail if request_provider does not match a configured integration'), and provides guidance on trace stitching. However, it does not explicitly state when not to use or compare to alternative logging methods.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate it's a mutation (readOnlyHint: false). The description confirms it modifies user fields and adds the important constraint that email and workspace roles are off-limits. It does not describe side effects, error conditions, or permissions, but given annotations already signal mutability, the description adds sufficient behavioral context for a simple update.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Delivers maximum information in two sentences: first sentence states the action and scope, second sentence specifies exclusions and suggests an alternative. No wasted words; front-loaded with essential details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with an output schema, the description covers what is updatable, what is not, and directs to an alternative. It lacks mention of prerequisites (e.g., user existence, authentication) or error behavior. However, given the tool's low complexity and the presence of openWorldHint, it is fairly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for each parameter. The overall description adds value by clarifying which parameters are and aren't editable, providing context beyond individual parameter descriptions. This compensates for the baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'Update', resource 'user', and lists specific fields (first name, last name, organization role). Explicitly distinguishes from update_workspace_member by stating what is not editable. This leaves no ambiguity about the tool's purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use and when-not-to-use guidance: 'Email and workspace roles are not editable here; use update_workspace_member for workspace membership changes.' Directly contrasts with a sibling tool, helping an agent select the correct tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the destructiveHint annotation, the description discloses key behaviors: the export is 'permanently stops', takes effect 'immediately', and 'does not roll back already-processed rows'. It also states the 403 response on non-Enterprise plans, which is valuable context not in annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, front-loaded with the purpose. Each sentence adds distinct value: what it does, behavioral consequences/retry, and access limitations. No fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the destructive nature, the description covers the essential context: what it cancels, permanence, no rollback, how to retry, and access restrictions. With an output schema present, return values need not be explained. This is complete for a cancellation mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already provides 100% coverage with a description for export_id ('The unique ID of the log export to cancel'). The tool description adds no additional parameter semantics, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource: 'Cancel a pending or running log export job.' It also distinguishes from siblings by explicitly naming start_log_export and delete_integration, making the tool's unique role clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states when to use this tool versus alternatives: 'unlike start_log_export which queues one or delete_integration which removes the source.' It also provides the retry workflow (create_log_export and start_log_export again) and enterprise access expectations, offering clear guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds behavioral context beyond annotations: irreversible, removes access immediately, cascades to servers. Annotations only mark destructiveHint=true; the description fills in the details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. Action and implications front-loaded, making it easy to quickly understand.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive delete operation, the description covers irreversibility, immediate impact, cascade, and a usage precaution. Output schema exists, so return values need not be described. Comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides a description for the single parameter 'id' (the integration ID or slug). The tool description does not add further parameter information, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Delete an MCP integration and all servers beneath it', using a specific verb and resource. It distinguishes from sibling delete operations like delete_mcp_server by noting the cascade effect.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: after confirming nothing depends on the integration, and warns of irreversibility and immediate access removal. No alternative tools are named, but the caution is sufficient for an irreversible delete.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description adds important behavioral context beyond annotations: permanence, immediate budget removal, and clearing tracked usage state. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states purpose, second adds critical behavioral context. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter and an output schema, the description is complete: it covers purpose, permanence, budget removal, and suggests a precheck. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with the 'id' parameter described. The description does not add extra meaning beyond the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Delete a usage limit by id.' It uses a specific verb and resource, and distinguishes itself from other delete tools on the server.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description explains when to use (delete a usage limit permanently) and advises checking list_usage_limit_entities first for impact, implicitly stating alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (destructiveHint=true), the description details the permanent nature of deletion and its consequences: removes memberships, revokes keys, ends sessions. This provides comprehensive behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first delivers the core purpose and effects, the second provides an alternative. It is concise and front-loaded with essential information, with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter, the description covers purpose, behavioral effects, and an alternative. The output schema exists but does not require additional description. The tool is fully specified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers the single parameter 'user_id' with a description. The tool description does not add additional semantics beyond the schema. Since schema coverage is 100%, the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'delete', the resource 'user', and the method 'by id'. It differentiates from the sibling tool 'delete_user_invite' by specifying that tool is for pending invites. The scope of deletion is thoroughly explained.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool (to delete a user by ID) and provides a direct alternative for a specific case ('use delete_user_invite for pending invites instead'). This is clear guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description adds valuable context beyond annotations (destructiveHint=true), explaining irreversible nature and secondary effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two efficient sentences, front-loaded with purpose, no extraneous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Simple tool with one parameter, description covers purpose, side effects, and return value adequately.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with adequate description for the single parameter; description adds no additional meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool deletes a virtual key by slug, distinguishing it from sibling delete tools for other resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly warns about irreversibility and impact on prompts/configs, advising to confirm no active dependencies first.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds important behavioral context: 'Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.' This discloses access restrictions and error outcomes. It also mentions pagination, which is not in annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, each serving a distinct purpose: what the tool does, when to use it vs alternatives, and enterprise restriction. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (31 params, 2 required, nested objects) and presence of an output schema, the description adequately covers return fields and pagination. It also mentions the enterprise gating, which is critical for usage. The description is sufficient for an agent to understand the tool's behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all 31 parameters have descriptions in the input schema. The description does not add parameter-specific meaning beyond summarizing the high-level purpose. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'paginated per-model breakdown'. It specifies the returned fields (total_groups, group_count, models array with request count, cost, token usage). It also distinguishes from siblings get_token_analytics and get_cost_analytics by mentioning their alternative use case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use this tool ('compare model cost, popularity, and efficiency') and when not to ('use get_token_analytics or get_cost_analytics for time-series trends instead'). It names specific alternative tools, providing clear guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, non-destructive, idempotent, and open-world. Description adds value by specifying what settings are returned, which goes beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. Essential information is front-loaded: action, resource, returned fields, and prerequisite.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, annotations, and output schema presence, the description covers all necessary aspects. It explains what the tool returns and how to obtain the required parameter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter (slug) with 100% schema description coverage. Description adds context about where to find the slug (URL or list_configs response), enhancing the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and resource ('config by slug'), and specifies the returned data: 'routing, cache, retry, and target settings'. It differentiates from sibling tools like list_configs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states prerequisite: 'Requires a known slug' and directs to list_configs for discovery. Provides clear when-to-use and alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint. The description adds meaningful context: the included fields are conditional ('when available'), and for path_format v2 the caller must provide created_at. No contradiction exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core action, and every clause contributes relevant information. It is appropriately concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-log retrieval with an output schema and strong annotations, the description covers essential behavior and the key path_format caveat. The output schema handles return format, so no further detail is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, providing a baseline of 3. The description adds the critical relationship that path_format v2 requires created_at, which is not marked required in the schema. It does not cover the 'type' parameter, but the schema fully documents it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get one gateway request log by ID' with a specific verb, resource, and scope, and lists the included payload elements (request, response, usage, cost, metadata). It distinguishes from sibling log-export tools by focusing on single-log retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool: when retrieving a specific gateway log by ID with full details. However, it does not explicitly name alternatives or provide exclusions, so it falls short of a perfect score.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint=false, but the description adds significant behavioral context beyond those: required API key scopes (logs.export, completion scope for some fields), enterprise gating, and the 403 error on non-Enterprise plans. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each earning its place: primary purpose, usage directive, and necessary caveats. Front-loaded with the core function, no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema is present and annotations cover operational hints, the description fully rounds out context: it explains when to call (before exports), what scopes are needed, and enterprise gating. This is complete for a read-only, single-parameter tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter workspace_id is fully described in the schema ('Workspace ID whose export field restrictions should be read'), with 100% schema description coverage. The description does not add syntax or additional meaning to the parameter beyond what the schema already provides, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb ('Get') and a precise resource ('organisation-managed fields that a workspace is restricted from including in log exports'). It clearly distinguishes the tool from sibling tools by referencing create_log_export and update_log_export, making its unique function obvious.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs when to use: 'Use before create_log_export or update_log_export to avoid requesting disallowed fields.' It also provides contextual caveats (API key scope, enterprise gating, 403 on non-Enterprise), giving clear guidance on prerequisites and expected failures.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, destructiveHint, idempotentHint, and openWorldHint. The description adds valuable behavioral context by specifying what is returned (full Portkey-side config including auth type, transport, masked keys) without contradicting annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the main purpose, and every word contributes. No extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one required parameter, output schema present), the description fully covers what the tool does, what it returns, and how it relates to a sibling tool. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already has 100% coverage with a description including 'ID or slug'. The description reiterates the same information without adding new parameter semantics beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Retrieve' and the resource 'one MCP integration' with identifier id or slug. It distinguishes from the sibling tool 'get_mcp_integration_metadata' by specifically mentioning that alternative for server-reported metadata.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance is provided: 'use get_mcp_integration_metadata for the server's self-reported metadata'. This clearly tells when to use this tool versus an alternative, making the usage context unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive. Description adds that it returns self-reported metadata with specific fields, enhancing agent understanding beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no fluff, efficient communication of purpose and distinction.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval tool with output schema and clear annotations, the description covers purpose, return values, and sibling differentiation. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and description does not add any additional meaning to the single 'id' parameter beyond what the schema already provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it retrieves external MCP server metadata, lists returned fields, and explicitly distinguishes from sibling tool get_mcp_integration for connection config.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Directly tells when to use this tool vs. get_mcp_integration, providing a clear alternative for a different purpose.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark this as read-only, open-world, and idempotent. The description adds valuable context on return semantics (USD cents per token or provider-specific unit), the types of metadata included (cache, audio, image, fine-tuning, calculation), and the exclusion of negotiated pricing. This goes beyond the structured annotations and helps the agent set expectations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences long, front-loaded with the core purpose, and every sentence adds critical information. There is no redundancy or filler, making it easy for an agent to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 params, read-only, output schema present), the description provides complete context: what it returns, its units, content categories, what it excludes, and when to use it. The output schema covers return structure, so no further detail is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description does not add parameter-specific meaning beyond the schema, but it reinforces the 'exact pair' requirement. Since the schema already handles parameter semantics fully, this is sufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource ('Get Portkey's current public pricing configuration') and clearly scopes it to one exact provider/model pair. This distinguishes it from any analytics tools or other pricing-related operations. The purpose is unambiguous and immediately understandably.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use the tool ('before setting integration pricing_adjustments or custom-model pricing') and what it does not do ('does not return your negotiated integration multiplier or require Portkey authentication'). This gives clear usage boundaries and prevents misuse, even without naming alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the safety profile is covered. The description adds valuable behavioral context by specifying auth scope, enterprise-gating, and the 403 response on non-Enterprise plans, plus clarifying that per-workspace exclusions are not included. It doesn't contradict annotations and adds useful information beyond them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, front-loaded with the core purpose, and each sentence earns its place: purpose, usage, exclusions, auth, and error behavior. No redundant or extraneous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters) and presence of an output schema, the description is complete: it specifies purpose, usage context, exclusions, access prerequisites, and error behavior. It fully equips an agent to select and invoke this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so per rubric the baseline is 4. The description doesn't need to add parameter details since there are none, and the schema coverage is effectively complete for an empty parameter set.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get the organisation-wide input and output guardrails that workspaces inherit by default' with a specific verb and resource, and explicitly distinguishes from related tools like update_organisation_defaults and exclusion list tools. It fully clarifies scope, making it unmistakable what this tool does relative to siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: 'Use this before update_organisation_defaults or when auditing baseline enforcement' and states what is excluded and where to find it ('directional exclusion list tools'). Also includes required scope and enterprise-gating requirements, giving clear when-to-use and when-not-to-use context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds value by specifying the exact return fields (budget, threshold, grouping, reset details), making the tool's behavior fully transparent without contradicting annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: the first front-loads purpose and return details, the second provides usage guidance. No unnecessary words; every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (1 parameter, no nested objects) and the existence of an output schema, the description adequately covers what the tool does, what it returns, and when to use it. No gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single parameter 'id'. The description does not add any new meaning beyond what the schema already provides, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get one usage limit by id') and specifies the returned data ('full budget, threshold, grouping, and reset details'). It distinguishes itself from sibling tool list_usage_limits by referencing it for discovery.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use list_usage_limits to discover ids or compare policies first,' providing clear when-to-use and when-not-to-use guidance with a named alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, idempotent, non-destructive. Description adds key behavioral context: 'Enterprise-gated. Returns 403 on non-Enterprise Portkey plans', which is beyond annotations and critical for agent awareness.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences. First front-loads purpose and resource, then provides usage guidance and constraints. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, return fields, usage guidance, alternatives, and enterprise constraint. Output schema exists, so return format is already detailed. Complete for a list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. Description does not add extra parameter details, but baseline is 3 since schema already covers meaning adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'List accepted org users with id, name, email, role, and timestamps', specifying the verb and resource. Distinguishes from sibling tools like list_user_invites and mentions usage context for get_user, update_user, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when to use: 'find a user_id before get_user, update_user, delete_user, or add_workspace_member' and points to alternative 'use list_user_invites for pending invitations'. Also notes enterprise gating and 403 error on non-Enterprise.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses pagination, return fields (actor, resource, metadata, timestamps), and enterprise restriction, adding value beyond annotations which already indicate read-only, idempotent, non-destructive behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with core purpose, followed by usage differentiation and limitation—every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given comprehensive parameters, annotations, and output schema, the description covers purpose, usage context, and limitations completely, leaving no critical gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with self-describing parameter names and types; description does not add parameter-level details but context about return fields aids understanding. Baseline score appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'List audit log events' with specific resource scope, and explicitly distinguishes from analytics tools by mentioning individual events vs aggregates.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use guidance ('use this instead of analytics when you need individual events, not aggregates') and notes enterprise-gating with 403 error for non-Enterprise plans.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is clear. The description adds behavioral details about paginated results (total, has_more) and the purpose of discovering IDs, which is valuable context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences. The first states the core function, the second provides return structure and sibling tool references. No extraneous words, well-organized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and full parameter descriptions, the description covers all necessary context: what the tool does, what it returns (paginated records with total/has_more), and how to get more detail from siblings. Complete for a list endpoint.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with all three parameters having descriptions. The description does not add significant new parameter information, as the schema already documents page_size, current_page, and workspace_id adequately. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists MCP integrations in the organization and distinguishes it from siblings like get_mcp_integration and list_mcp_servers, which are explicitly named for specific use cases.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool (to discover integration IDs via pagination) and when to use alternatives (get_mcp_integration for full config, list_mcp_servers for servers under an integration).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, but the description adds valuable behavioral context: it 'reads output exclusions only', states the required API key scope, emphasizes enterprise gating, and documents the 403 error on non-Enterprise plans. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each carrying substantive information (purpose, usage context, permissions/error behavior). No redundant phrasing. Well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity, the description covers purpose, usage scenarios, exclusions vs. defaults, authentication scope, enterprise restriction, and error condition. An output schema exists, so return values need no further explanation. Description is complete for tool selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear description for organisation_id. The description reinforces the parameter by saying 'for one organisation', but does not add significant new semantic details beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List workspaces excluded'), the resource ('organisation-wide output guardrails'), and the scope ('for one organisation'). It distinguishes from the sibling tool list_input_guardrail_workspace_exclusions by explicitly specifying 'output guardrails' and from the matching update tool by using 'list' and 'reads'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: 'Use this to audit exceptions or establish the current state before the matching update tool'. It also clarifies what the tool does not return ('does not return the organisation's default guardrail list'), differentiating it from get_organisation_defaults, and notes enterprise gating and 403 behavior.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds meaningful behavior beyond annotations: clarifies the read scope ('reads mappings only'), discloses enterprise gating and the 403 error on non-Enterprise plans, and notes pagination for large directories. No contradiction with readOnlyHint or idempotentHint.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise, information-dense sentences. Front-loaded with purpose, followed by usage guidance and behavioral caveats. No redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers purpose, usage triggers, exclusions, authentication requirements, error behavior, filtering, and pagination. With an output schema present, return values need no explanation. The tool is fully contextualized for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description mentions filtering by workspace, group, or role and pagination, but the schema already provides detailed parameter descriptions. No additional semantic value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'List identity-provider SCIM group mappings that automatically grant Portkey workspace roles.' It clearly distinguishes from sibling tools like list_scim_groups by specifying 'mappings' and the role-granting behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: 'audit provisioned access or obtain mapping_id before delete_scim_workspace_mapping.' It also clarifies what it does not do ('does not query individual workspace members') and mentions enterprise gating, giving clear context versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description clarifies that the tool 'sends a fresh email without modifying the invite record, expiry, or role', which complements the annotations (readOnlyHint=false, destructiveHint=false) by specifying the non-destructive nature despite being a mutation. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that efficiently deliver purpose, behavior, usage differentiation, and prerequisite tool recommendations without any redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple parameter set and comprehensive annotations, the description provides complete context including behavior, side effects (none), and usage hints. No gaps remain for an agent to safely invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (single parameter invite_id with clear description). The tool description does not add further semantic meaning beyond what the schema already provides for the parameter, thus baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Resend the email for a pending invite that has not been accepted') and contrasts with sibling tool 'invite_user' which creates a new invite, making the purpose and distinction unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use this tool (for resending pending invites) and when not (use invite_user to create new invites). Additionally, recommends calling get_user_invite first to verify invite existence and list_user_invites to discover IDs, providing excellent usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behavioral traits beyond annotations: asynchronous queueing, no immediate rows/download return, dependency on prior job creation, enterprise-only gating with 403 on non-Enterprise plans. This adds significant context not present in annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences, each adding essential information: action, async behavior, alternative tools, and enterprise restriction. No redundant phrases or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple one-parameter schema, existing output schema, and rich annotations, the description covers all critical aspects: what it does, how it behaves asynchronously, what it does not return, next steps, and error condition. Fully sufficient for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the single parameter export_id is already well-described in the schema. The description reinforces that it must be a previously created job, but this adds minimal new semantic value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action: 'Start processing a previously created log export job.' It distinguishes itself from siblings like create_log_export and get_log_export by emphasizing that this is the start step, not creation or polling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly provides usage context: it operates on a previously created job, is asynchronous, and directs the agent to use get_log_export for polling and download_log_export after completion. Enterprise gating is also noted, covering when it is and isn't applicable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
It discloses important non-obvious behavior: each call creates a new version instead of overwriting, changes apply immediately to all API keys and prompts, only provided fields change, and the slug is immutable. With only false annotations, this description carries the full burden of safety and semantics, and it does so comprehensively.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every sentence earns its place: versioning behavior, editable fields and slug constraint, immediate effects and prerequisite workflow, and return value. It avoids fluff while covering all critical operational details for a complex update tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (versioning, partial updates, global side effects) and the availability of an output schema, the description provides all necessary context: how to prepare, what changes, what remains fixed, how to recover, and what is returned. No significant gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds valuable semantics beyond the schema by clarifying partial-update behavior ('Only provided fields change') and the immutability of the slug. It also enumerates the editable settings, reinforcing which parameters are meaningful for an update call.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Update a config by slug', which is a specific verb+resource statement. It distinguishes this from create_config by noting slug stays fixed and every call creates a new version, and from delete_config by the recoverability of old versions. The editable field groups are enumerated, making the scope clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context by instructing to get the slug from list_configs and review current settings with get_config before editing. It also mentions list_config_versions for recovery of earlier versions. However, it does not explicitly name create_config as the alternative for creating a new slug, so the when-not-to-use guidance is only implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses merge vs. override behavior, immediate enforcement changes, idempotency of repeating states, authentication requirements, and 403 errors on non-Enterprise plans. This significantly enriches the annotations (readOnlyHint=false, destructiveHint=true, idempotentHint=true), which only hint at these behaviors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and information-dense. Six short sentences each contribute unique value: purpose, entry semantics, merge behavior, guidance, auth, and error condition. No filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a moderately complex mutation tool, the description covers all essential context: auth scope, Enterprise gating, immediate enforcement impact, merge vs. override semantics, and idempotency. With annotations and an output schema present, this description is fully sufficient for correct tool selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by explaining that each entry excludes or restores one workspace and that default merge preserves unmentioned workspaces, which clarifies the workspaces array and override_existing parameter beyond schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool sets workspace exclusions for organisation-wide input guardrails, with a specific verb and resource. It distinguishes itself from the sibling update_output_guardrail_workspace_exclusions by specifying 'input guardrails' and explains the core entry semantics.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises reviewing the matching list tool first because enforcement changes immediately, and specifies required API key scope and Enterprise plan gating. While it doesn't name alternatives, the context strongly implies the input/output distinction and provides clear preconditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (idempotentHint, readOnlyHint=false), the description discloses immediate enforcement propagation, partial-update behavior, required API key scope, Enterprise gating, and the 403 error on non-Enterprise plans. This adds substantial operational context that annotations alone do not convey. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact—four sentences—with each sentence earning its place: function, behavior, safety, permissions, and error handling. It is front-loaded with the core purpose and avoids redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a moderately complex mutation tool, the description covers all essential aspects: what it does, how it behaves (immediate propagation), prerequisites (inspect read tools), idempotency, authentication, and error condition. The existence of an output schema also covers return values. No significant gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds meaningful semantics by stating "Only supplied directions change," clarifying that parameters are optional and act as partial updates. This is extra value beyond the schema descriptions, warranting a slightly higher score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb "Replace" and clearly identifies the resource: "organisation-wide default input and/or output guardrail lists inherited by workspaces." It distinguishes itself from sibling tools like update_input_guardrail_workspace_exclusions and update_output_guardrail_workspace_exclusions by focusing on the defaults rather than exclusions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description instructs the agent to "inspect get_organisation_defaults and the directional workspace exclusions first," providing clear prerequisite steps. It also notes that only supplied directions change and that repetition is safe, giving practical usage context. It stops short of explicitly naming alternatives or when-not-to-use scenarios, but the guidance is solid.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes well beyond the annotations (which already mark destructiveHint=true and idempotentHint=true) by disclosing concrete behavioral traits: enforcement changes immediately, override_existing replaces prior states while default merge preserves unmentioned workspaces, requires an organisation service API key with organisation_exclusions.update scope, is Enterprise-gated, and returns 403 on non-Enterprise plans. It also clarifies that repeating the same states is safe. No contradiction with annotations is present.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every sentence earns its place. It covers purpose, usage prerequisite, behavioral side effects, idempotency, authentication scope, enterprise gating, and expected error response—all in five concise sentences. The structure is front-loaded with the core action, followed by key behavioral nuance, then supporting operational details. No redundant or filler content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a potentially destructive mutation tool with an output schema present, the description is comprehensive. It covers the tool's side effects (immediate enforcement), merge/override semantics, idempotency, required auth scope, enterprise gating, and a specific error condition (403). The presence of an output schema means return-value details need not be in the description. The combination of schema coverage, annotations, and description provides a complete picture for an agent to invoke this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for all parameters, which would normally set a baseline of 3. The description adds meaningful semantic detail beyond the schema: it explains that 'override_existing replaces prior states while the default merge behavior preserves unmentioned workspaces', which clarifies the interaction between the override_existing boolean and the workspaces array. This goes beyond the schema's per-parameter descriptions, so a 4 is warranted.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Set workspace exclusions from organisation-wide output guardrails', which is a specific verb+resource statement. The tool name includes 'output_guardrail_workspace_exclusions' and there is a sibling 'update_input_guardrail_workspace_exclusions', so the explicit 'output guardrails' wording clearly distinguishes it from the input-guardrail alternative. The purpose is unambiguous and actionable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly instructs to 'Review the matching list tool first because enforcement changes immediately', providing a clear prerequisite and warning about side effects. It also notes that 'Repeating the same states is safe', which guides usage around idempotency. It does not explicitly state 'use this instead of the input guardrail tool', but the purpose clarity and context make the usage context well-defined.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are present and consistent (readOnlyHint=false, destructiveHint=false, idempotentHint=false, openWorldHint=true). The description adds that only provided fields change and that the new version is inactive until published, which are important behaviors not covered by annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, each adding value. The purpose is front-loaded, and there is no redundant or unnecessary text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, the description does not need to explain return values. It covers the key behavioral context (partial update, new inactive version) and is complete for this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all parameters. The description adds value by stating 'Only provided fields change', clarifying partial update semantics, which goes beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (create) and resource (a new version of a partial), and implies the resource is a partial, distinguishing it from sibling tools like publish_partial that make a version current.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that only provided fields change and the new version stays inactive until publish_partial is used, providing clear context for when to use this tool and hinting at the alternative publish tool. However, it does not explicitly say 'use publish_partial to make it current'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description adds behavioral context beyond annotations: it creates a version immediately and returns ids. Annotations already indicate non-readonly, non-destructive, non-idempotent, so no contradiction. Some specifics like 'creates a new version immediately' are helpful but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences are compact and front-loaded with purpose, then usage guidance, then parameter details. Every sentence adds unique value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (15 parameters, nested objects), the description covers core workflow, tool selection, parameter guidance, and return values. Output schema exists, so return details are sufficient. No gaps for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 100% schema coverage, description adds valuable meaning: distinguishes between 'string' (legacy) and 'messages' (structured), recommends 'messages' for multi-message prompts, and explains that messages are serialized. This aids correct parameter selection beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Create a new prompt template and initial version' and distinguishes from sibling 'migrate_prompt' for CI/CD flows. The verb 'create' with resource 'prompt template' is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises use for first-time setup versus migrate_prompt for idempotent flows. Also explains when to use plain text vs structured messages, providing clear context for parameter selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already set `idempotentHint=true` and `destructiveHint=false`. The description goes beyond by explaining that 'nothing is overwritten' and that if content matches, it 'no-ops'. It also details the dry_run behavior ('reports what would happen without changing anything') and what the tool stores in template_metadata (app/env). This adds valuable behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, dense paragraph that wastes no words. Each sentence adds value: purpose, contrast with sibling, behavior, dry_run, metadata, return fields. It is front-loaded with the most critical information and remains brief given the tool's complexity (15 params).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's high parameter count and nested objects, the description covers the core idempotent behavior, dry_run, key parameters, and return fields. It mentions that `messages` are serialized to legacy string format but does not detail the exact serialization process. However, the output schema likely describes return values. Minor gap, but overall sufficient for agent selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds meaning by noting that `collection_id` is used for lookup and creation, `dry_run` controls dry-run mode, and `template_metadata` stores app/env. However, it omits explanation for some parameters like `model`, `tools`, `messages`, and `string`. Still, the provided details enhance understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Create or update a prompt in one idempotent step for CI/CD and prompt-as-code flows'. It immediately distinguishes from `create_prompt`, which always creates a new prompt. The verb 'migrate' combined with the explanation of idempotent upsert behavior makes the purpose precise and actionable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use this tool versus the sibling `create_prompt` ('unlike create_prompt which always makes a new prompt'). It also explains the lookup logic by name within collection_id, the three possible outcomes (create, update, no-op), and the dry_run flag for safe testing. It directs the user to `list_collections` for obtaining collection_id.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses important behavioral traits beyond annotations: 'At least one setting is required, new configs become active immediately once referenced by a key or prompt, and the call returns the new id and version_id.' This adds activation semantics and return value details, giving the agent clear expectations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, and every clause earns its place. The structure is efficient with no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 8-parameter schema with full descriptions and an output schema (so return values are already documented), the description covers all necessary contextual aspects: what it does, when to use it, key behavioral rules, and activation implications. Nothing critical is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds a non-obvious constraint ('At least one setting is required') not enforced by the schema (which only requires name), and mentions return fields (id, version_id), providing extra semantic context beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Create a config that defines routing, cache, retry, and targets for requests.' It uses a specific verb (create) and resource (config), and distinguishes itself from siblings by referencing update_config and list_config_versions as alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly provides usage guidance: 'use update_config to modify an existing one and list_config_versions for history.' This tells the agent when to use create_config versus related tools, and also implies it is for new configs only.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate a mutating operation (readOnlyHint=false), but the description adds side-effect context: it provisions access for current and future members, can pre-create the group via scim_group_name, and may return 403 on non-Enterprise plans. This goes well beyond the structured fields and provides essential behavioral expectations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, each adding distinctive information: core purpose, parameter constraint/pre-creation, discovery tool references, side-effect/enterprise-403 warning. No redundant filler or repetition of schema details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex SCIM mapping operation with an output schema present, the description covers purpose, prerequisites, parameter selection rule, side effects, and error condition. It leaves return-value documentation to the output schema, which is acceptable. No significant gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents all four parameters with descriptions covering the one-of constraint. The description reinforces the exactly-one rule and adds the pre-creation behavior for scim_group_name, which is not in the schema. Given 100% schema coverage, this extra nuance elevates the score above baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+object+effect: 'Map one identity-provider SCIM group to a Portkey workspace role so current and future group members receive access automatically.' It clearly distinguishes from sibling `add_workspace_member` by stating it grants access via group rather than individually.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly names prerequisite discovery tools (list_scim_groups, list_workspaces) and compares to the alternative add_workspace_member, stating when this tool is appropriate ('distinct from... grants one user directly'). Also notes Enterprise-gated plans.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (destructiveHint, idempotentHint), the description adds critical behavior: existing provisioned members remain, Enterprise-gating, and 403 error on non-Enterprise plans. This gives the agent important context to set expectations and handle errors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is tightly written in three sentences, each providing unique, necessary information. It is front-loaded with the primary action, followed by consequential details, with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the single parameter, rich annotations, and presence of an output schema, the description covers all relevant context: purpose, side effects, prerequisites, exclusions, gating, and error behavior. It is fully self-sufficient for an agent to invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds value by specifying that mapping_id comes from list_scim_workspace_mappings. This reinforces the parameter's origin, though it could include more details about required format or validation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Delete') and resource ('SCIM group-to-workspace mapping'), clearly distinguishing it from sibling tools like create_scim_workspace_mapping and list_scim_workspace_mappings. It also states the key side effect (stopping future group updates), making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly instructs to 'Inspect list_scim_workspace_mappings first', providing a clear prerequisite and source for the mapping_id. It also clarifies exclusions (does not delete the identity-provider group or the Portkey workspace), helping the agent avoid misuse.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds meaningful behavioral context beyond annotations: it states the action 'immediately ends that connection but does not revoke future access,' clarifying the scope of destructiveness. It also explains key derivation for user keys, supplementing the destructiveHint and idempotentHint annotations without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each earning its place: core action, scope/alternative, and parameter guidance plus a best-practice tip. No redundancy, well front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with three parameters (one required) and an output schema, the description fully covers purpose, exclusions, auth requirements, and a pre-check suggestion. Nothing essential is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds valuable context: 'Provide user_id with service keys (user keys derive it), and workspace_id with organisation admin keys.' This explains the conditional necessity of optional parameters, going beyond the static schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description begins with a specific verb and resource: 'Disconnect one user's active session from a Portkey-managed MCP server.' It clearly distinguishes from the sibling tool update_mcp_server_user_access, which is for removing access rather than just ending an active session.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance is given: 'use update_mcp_server_user_access when access itself should be removed' and 'Inspect list_mcp_server_connections first when the target is uncertain.' It also specifies key-type-dependent parameter usage, making it clear when to provide user_id vs workspace_id.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark it read-only, idempotent, and non-destructive, but the description adds useful behavioral details: it reads live connection state, lists fields returned, and explicitly excludes upstream reachability testing. It also explains permission scoping for different API key types, which goes beyond the structured annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded with the core action, then covers use cases, access control, and limitations in just three sentences. Every sentence contributes unique value with no repetition of schema details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists and annotations convey safety, the description is complete: it covers what is returned, when to use it, who can use it with which key types, and what it does not do. There are no significant gaps for an agent to select and invoke this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already provides 100% parameter coverage with defaults and filtering semantics. The description adds meaning beyond the schema by explaining that service keys can list all users, user keys default to their own user, and org admin keys must provide workspace_id—this supplements the raw schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'List active connection records for one Portkey-managed MCP server,' including user, connected state, and timestamps. This distinguishes it from sibling tools like list_mcp_servers and test_mcp_server, which cover different server details or connectivity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly states when to use the tool ('audit sessions or identify the user/workspace before disconnect_mcp_server_connection') and provides an alternative for a different need: 'does not test upstream server reachability (use test_mcp_server for that).' It also clarifies key-scope behavior (service keys vs. user keys, org admin workspace_id requirement).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds valuable behavioral context beyond annotations, specifying that only name/description can be updated and that it does not affect membership or prompts. Annotations already indicate non-destructive and non-read-only, so the description complements without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words. The first sentence states the action and scope; the second provides exclusions and usage guidance. Front-loaded and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple update tool with 3 parameters and an output schema, the description is complete. It covers what is updated, what is not, and when to use the tool. No gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds meaning by stating 'name or description only', confirming the intended parameters. While the schema lists them, the description reinforces the limitation, adding some value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update a collection's name or description only', specifying the verb (update), resource (collection), and scope (name or description only). It effectively distinguishes from sibling tools like update_prompt or update_workspace by highlighting its limited scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'use it for metadata changes rather than reorganization.' It also clarifies what the tool does not do ('does not move prompts or change membership'), helping the agent avoid misuse.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
While annotations already flag destructiveHint and readOnlyHint, the description adds vital behavioral context: partial updates are honored, changes to key/secret/config are immediately effective and can disrupt live traffic, pricing adjustments modify cost analytics, and related concerns are deliberately isolated. This materially enriches what the bare annotations convey and aligns with them (no contradiction).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences deliver purpose, side effects, usage safety, and sibling differentiation without any filler. The most important behavioral caveat (disruption risk) is front-loaded, and every clause contributes information an agent needs before invoking the tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's 15 parameters, rich schema, and existing output schema, the description covers purpose, mutation semantics, immediate effects, risk profile, and separation from closely related tools. It leaves no major gap: an agent knows what to call, what to expect, and how to avoid unintended consequences. The explicit pointer to get_integration for pre-review further rounds out operational context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already covers all 15 parameters with detailed per-field descriptions, so the baseline is a 3 per the rubric. The description adds value by grouping fields into behavioral categories (e.g., 'key, secret mapping, and config' vs. 'pricing adjustments') and clarifying partial update semantics ('Only provided fields change'), which helps the agent understand how parameter combinations behave even though individual properties are schema-documented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb ('Update') and an explicit resource ('an integration'), then enumerates the exact updatable fields: name, description, API key, provider config, Secret Reference mappings, and pricing adjustments. It further distinguishes this tool from siblings by naming update_integration_models and update_integration_workspaces as separate concerns, making its scope unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives concrete guidance: only provided fields change, key/secret/config changes take effect immediately and can disrupt providers or live requests, and pricing multiplier changes affect cost analytics. It also instructs the agent to review get_integration first and explicitly redirects model/workspace access changes to separate tools, providing both usage context and exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds meaningful behavioral details beyond annotations: only specific fields can be changed after creation, the operation is suitable only before or between runs, and it returns 403 on non-Enterprise plans. This goes beyond the basic readOnly/destructive flags and helps the agent anticipate side effects and limitations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences pack all essential information: purpose, timing, mutable fields, alternative tools, and a critical error condition. No filler or redundant repetition of schema details. Front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the moderate complexity, complete parameter schema, and presence of an output schema, the description covers all necessary context: what can be changed, when to call it, what to do before/after, and a failure mode. An agent has enough to decide when and how to invoke this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by explicitly identifying which parameters are mutable (workspace_id, time_of_generation_max, requested_fields) and implying export_id is the required target. This contextualizes parameter usage beyond the raw schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates an existing log export configuration, with a specific verb ('Update') and resource ('log export configuration'). It also distinguishes itself from sibling tools by specifying timing ('before or between export runs') and pointing to alternatives like get_log_export and start_log_export.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit usage guidance: which fields are mutable, when to use it (before or between export runs), and exactly when to use other tools ('use get_log_export to review the current job and start_log_export after the definition is ready'). It also notes Enterprise-gating and 403 behavior, giving clear context for tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses critical behavioral traits beyond annotations: partial update semantics ('Only supplied fields change'), immediate effect and potential to break active clients for URL/auth/header/secret changes, and the need to inspect first. This adds significant value beyond the destructiveHint annotation and does not contradict it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, each with a distinct purpose: what the tool does, key behavioral warnings, and alternative tool guidance. It is front-loaded with the core action and fields, and contains no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (8 params, nested objects) and the existing output schema, the description covers all essential aspects: purpose, partial update behavior, immediate impact warnings, prerequisite inspection, and alternative tooling. It is fully sufficient for an agent to use the tool safely and correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% so all parameters are documented, but the description adds important semantic context about partial updates ('Only supplied fields change') and groups fields by impact, which helps the agent understand how parameters combine. This elevates it above the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update') and resource ('MCP integration'), lists the specific updatable fields, and distinguishes itself from the sibling tool update_mcp_server by explicitly naming the alternative for server display metadata changes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides explicit guidance on when to use this tool versus the alternative ('Use update_mcp_server when changing only a Portkey server instance's display metadata') and advises inspecting get_mcp_integration first, giving clear contextual usage instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are basic (non-readonly, non-destructive). The description adds that changes take effect immediately for all tagged versions and do not reassign labels or touch history, offering valuable behavioral context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, followed by behavioral notes and prerequisite. No redundant information; every phrase adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 4 parameters (1 required) and an output schema, the description covers immediate effects, side effects, and prerequisite steps. No gaps given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions. The description reinforces that only name, description, or color can be updated, adding operational context. While not adding new parameter details, it clarifies mutation scope.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update a prompt label's name, description, or color only', specifying the verb and resource. It distinguishes from sibling update_prompt_version, which changes label assignments, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly contrasts with update_prompt_version and advises using list_prompt_labels to find the label_id first, providing clear when-to-use and prerequisite instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![portkey-admin-mcp MCP server](https://glama.ai/mcp/servers/CodesWhat/portkey-admin-mcp/badges/card.svg)](https://glama.ai/mcp/servers/CodesWhat/portkey-admin-mcp)

Score Badge

Copy to your README.md:

[![portkey-admin-mcp MCP server](https://glama.ai/mcp/servers/CodesWhat/portkey-admin-mcp/badges/score.svg)](https://glama.ai/mcp/servers/CodesWhat/portkey-admin-mcp)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/CodesWhat/portkey-admin-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server