Paddle MCP Server

Official

by PaddleHQ

Overview Schema Related Servers Score Discussions

TypeScript

Remote

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v1.0.0
Disambiguation5/5
Each tool has a clearly distinct purpose targeting specific resources and actions in the Paddle ecosystem. For example, create_address, get_address, and list_addresses are distinct from create_adjustment, get_adjustment_credit_note, and list_adjustments, with no overlap in functionality. The descriptions provide clear boundaries, making it easy for an agent to differentiate between tools like create_transaction and preview_transaction_create.
Naming Consistency5/5
Tool names follow a highly consistent verb_noun pattern throughout, such as create_address, get_address, list_addresses, and preview_transaction_create. All tools use snake_case uniformly, with verbs like create, get, list, preview, replay, and update applied predictably across different entities like customer, subscription, and notification. This consistency aids in readability and predictability.
Tool Count2/5
With 63 tools, the count is excessive for typical MCP server purposes, making it overwhelming and heavy for agents to navigate. While Paddle's API is comprehensive, this many tools suggests poor scoping for an MCP interface, where a more streamlined set (e.g., 15-25 tools) would be more appropriate. The high count increases cognitive load and potential for misselection.
Completeness5/5
The tool set provides complete CRUD and lifecycle coverage for Paddle's domain, including entities like addresses, adjustments, businesses, customers, discounts, notifications, prices, products, reports, simulations, subscriptions, and transactions. It includes create, get, list, preview, and update operations where applicable, with no obvious gaps—agents can perform all core workflows from creation to management and reporting without dead ends.
Average 3.9/5 across 63 of 63 tools scored. Lowest: 3.3/5.
See the Tool Scores section below for per-tool breakdowns.
- 1 of 3 community issues answered or closed in the last 6 months
- 0 commits in the last 12 weeks
- No stable releases found
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under Apache 2.0.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, confirming this is a non-destructive write operation. The description adds that it creates a new entity and includes a copy in the response if successful, which provides useful behavioral context beyond annotations. However, it doesn't detail error conditions, rate limits, or authentication requirements, leaving gaps in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the tool's purpose but includes verbose warnings about not fabricating details and asking for clarification, which are generic and could be condensed. It's moderately structured but has some redundancy, reducing efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters with 100% schema coverage, annotations covering safety, and no output schema, the description is adequate but not comprehensive. It explains the tool's purpose and success response but lacks details on failure modes, side effects, or integration with sibling tools, leaving room for improvement in context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are well-documented in the schema. The description doesn't add any specific parameter semantics beyond implying that all information should be gathered before calling. This meets the baseline of 3 since the schema carries the burden, but no extra value is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a new business entity for a customer in Paddle and explains what business entities are. It distinguishes from individual customers but doesn't explicitly differentiate from sibling tools like create_customer or get_business beyond the name. The purpose is specific but lacks direct sibling comparison.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied guidance: use when working with a business rather than an individual, and customers don't need a business to purchase. It warns against fabricating details and advises asking for clarification if ambiguous. However, it doesn't explicitly state when to use this vs. alternatives like create_customer or get_business, nor does it mention prerequisites beyond having information ready.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, which the description doesn't contradict. The description adds context about sub-entities (addresses, businesses) and the response format ('includes a copy of the new customer entity'), which provides useful behavioral information beyond annotations. However, it doesn't cover aspects like rate limits, authentication needs, or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is moderately concise but includes some redundant or overly general advice (e.g., 'Don't fabricate, imagine, or infer details'). The first two paragraphs are well-structured, but the latter part contains generic guidance that could be streamlined. It's front-loaded with purpose but could be more efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (creation operation, 4 parameters, no output schema), the description provides basic context about sub-entities and response format. However, it lacks details on error handling, idempotency, or system-specific constraints. With annotations covering safety but no output schema, the description is adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all four parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema, such as explaining relationships between parameters or providing examples. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool 'create a new customer in Paddle' and explains that 'Customer entities hold information about the people and businesses that make purchases.' This provides a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'create_business' or 'create_address' beyond mentioning their relationship.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides some usage guidance with 'Ensure you have all the information needed before making the call' and mentions related sub-entities (addresses, businesses) that can be created through sibling tools. However, it doesn't explicitly state when to use this tool versus alternatives like 'get_customer' or 'list_customers,' nor does it provide clear exclusions or prerequisites beyond general caution.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=false and destructiveHint=false, indicating this is a non-destructive write operation. The description adds that 'If successful, the response includes a copy of the new discount group entity,' which provides useful behavioral context about the return value. However, it doesn't mention authentication requirements, rate limits, or potential side effects beyond what annotations already cover.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is reasonably structured with purpose first, then usage context, then behavioral notes. However, the middle paragraph contains generic advice ('Don't fabricate, imagine, or infer details') that applies to all tools rather than being specific to this one, reducing efficiency. The core information could be conveyed more concisely without losing value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a single parameter with full schema coverage and annotations indicating a non-destructive write operation, the description provides adequate context about what the tool does and when to use it. However, without an output schema, the description only briefly mentions the response format. For a creation tool, more detail about what constitutes success/failure or the structure of the returned entity would be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage with a single 'name' parameter fully documented. The description doesn't add any parameter-specific information beyond what's in the schema, so it meets the baseline of 3. The general advice about 'Ensure you have all the information needed' doesn't provide specific parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool 'will create a new discount group in Paddle' and explains that discount groups 'are used to organize and manage related discounts under a group name.' This provides a specific verb (create) and resource (discount group) with context about its purpose. However, it doesn't explicitly differentiate from sibling tools like 'create_discount' or 'list_discount_groups' beyond mentioning the group concept.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance by stating 'Create one when managing multiple discounts together, like for a campaign, promotion, or team.' This gives context for when to use it but doesn't explicitly mention when NOT to use it or name alternatives (e.g., 'create_discount' for individual discounts). The additional text about ensuring information and not fabricating details is general advice rather than specific tool usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=true, which the description aligns with by using 'retrieve' (a read operation). The description adds minimal context about the data source ('Paddle') but does not disclose additional behavioral traits such as error handling, authentication needs, or rate limits. With annotations covering safety, a baseline 3 is appropriate for the slight added value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It front-loads the core action ('retrieve an address') and includes essential details (source, IDs), making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (simple retrieval), 100% schema coverage, and read-only annotation, the description is minimally adequate. However, without an output schema, it lacks details on return values or format, and it misses sibling differentiation, leaving gaps in full contextual understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear descriptions for both parameters. The description mentions using 'ID and related customer ID,' which mirrors the schema but adds no extra meaning about parameter relationships or usage. This meets the baseline when schema coverage is high.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'retrieve an address for a customer from Paddle using its ID and related customer ID.' It specifies the verb ('retrieve'), resource ('address'), and data source ('Paddle'), but does not explicitly differentiate it from its sibling 'list_addresses' or other get_* tools, which would require a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'list_addresses' or other retrieval tools. It mentions the required IDs but offers no context about prerequisites, error conditions, or typical use cases, leaving the agent without operational direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, so the agent knows it's safe. The description adds that it retrieves 'for a customer' and specifies the source (Paddle), which offers useful context beyond annotations. However, it doesn't mention error conditions, rate limits, or authentication requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with zero waste—directly states the tool's purpose without fluff. Efficiently front-loaded with the core action and key parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read operation with good annotations and full schema coverage, the description is adequate but minimal. It lacks output details (no output schema) and doesn't explain error handling or typical return structure, leaving gaps for agent understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both parameters clearly documented in the schema. The description mentions using 'ID and related customer ID' but doesn't add syntax, format, or relationship details beyond what the schema already states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('retrieve') and resource ('a business'), specifying it's from Paddle using ID and customer ID. It distinguishes from 'list_businesses' by focusing on single retrieval, but doesn't explicitly contrast with other get_ tools like 'get_customer' or 'get_address'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'list_businesses' or other retrieval tools. The description mentions needing both IDs but doesn't provide context about prerequisites or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description doesn't add behavioral context beyond what annotations provide. Annotations declare readOnlyHint=true, which already indicates this is a safe read operation. The description doesn't disclose additional traits like authentication requirements, rate limits, error conditions, or what happens if the customer ID doesn't exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise - a single sentence that directly states the tool's purpose with zero wasted words. It's perfectly front-loaded and appropriately sized for this simple retrieval tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read operation with good annotations (readOnlyHint) and full parameter documentation, the description is minimally adequate. However, without an output schema and with no behavioral context about what gets returned or error conditions, there are clear gaps in completeness for agent usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the input schema already fully documents the single 'customerId' parameter. The description doesn't add any meaningful semantic context beyond what's in the schema - it mentions retrieving by ID but doesn't explain format, validation, or examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('retrieve') and resource ('a customer from Paddle'), making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'list_customers' or 'get_customer' vs other 'get_' tools, which would be needed for a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. There's no mention of prerequisites, when to choose this over 'list_customers', or any context about its specific use case compared to other customer-related tools in the sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, so the agent knows this is a safe read operation. The description adds minimal context by specifying 'by its ID,' which clarifies the lookup mechanism. However, it doesn't disclose any additional behavioral traits like error handling, authentication needs, rate limits, or what happens if the discount ID doesn't exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it easy to parse. Every word earns its place without redundancy or unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read operation with one parameter and readOnlyHint annotation, the description is minimally adequate. However, without an output schema, it doesn't explain what data is returned (e.g., discount details, status, or error formats). Given the tool's simplicity, the description meets basic needs but lacks depth for full contextual understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the single parameter 'discountId' fully documented in the schema as 'Paddle ID of the discount.' The description adds no additional meaning beyond this, merely restating that retrieval is 'by its ID.' With high schema coverage, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('retrieve') and resource ('a discount from Paddle by its ID'), making the purpose unambiguous. However, it doesn't distinguish this tool from other 'get_' siblings like 'get_discount_group' or 'get_customer', which follow the same pattern of retrieving specific resources by ID.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With siblings like 'list_discounts' (for multiple discounts) and 'get_discount_group' (for related resources), the agent must infer usage from naming conventions alone. No explicit when/when-not instructions or prerequisite information is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations provide readOnlyHint=true, indicating a safe read operation. The description adds minimal context by specifying retrieval by ID, but does not disclose additional behavioral traits such as error handling, rate limits, or authentication needs. With annotations covering safety, a 3 is appropriate as the description adds some value but lacks rich behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It is appropriately sized and front-loaded, clearly stating the tool's purpose without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (single parameter, read-only), annotations covering safety, and no output schema, the description is minimally adequate. However, it lacks details on return values or error cases, which would be helpful for completeness. A 3 reflects a basic but incomplete description for this context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the parameter 'discountGroupId' fully documented. The description implies retrieval by ID but does not add meaning beyond what the schema provides, such as ID format or examples. Baseline 3 is correct when the schema handles parameter documentation effectively.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'retrieve a discount group from Paddle by its ID.' It specifies the verb ('retrieve'), resource ('discount group'), and source ('Paddle'), but does not explicitly differentiate it from sibling tools like 'list_discount_groups' or 'get_discount', which would require a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention sibling tools like 'list_discount_groups' for listing multiple groups or 'get_discount' for individual discounts, nor does it specify prerequisites or exclusions for usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds value by specifying that it retrieves from 'Paddle' and mentions the 'include' parameter for related entities, offering some behavioral context. However, it doesn't disclose additional traits like error handling, rate limits, or response format, which could be useful given no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and concise, with two sentences that directly address the tool's function and a key parameter. There is no wasted text, and information is front-loaded, making it efficient for an agent to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (2 parameters, 1 required) and annotations covering safety, the description is adequate but has gaps. It lacks output details (no schema provided), doesn't explain prerequisites like authentication, and offers no usage guidelines. This makes it minimally viable but incomplete for optimal agent use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents both parameters. The description adds minimal semantics by explaining that 'include' can fetch 'related entities' like 'product,' but this is largely redundant with the schema. Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'retrieve a price from Paddle by its ID.' It specifies the verb ('retrieve'), resource ('price'), and source ('Paddle'), which is specific and actionable. However, it doesn't explicitly differentiate from sibling tools like 'list_prices' or 'preview_prices,' which would require a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It mentions the 'include' parameter for related entities but doesn't explain when this is necessary or compare it to other tools like 'get_product' or 'list_prices.' This lack of contextual direction leaves the agent without usage criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, indicating a safe read operation. The description adds useful context about including related entities ('prices') in the response, which goes beyond annotations. However, it doesn't disclose other behavioral traits like error handling, rate limits, or authentication needs, leaving room for improvement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized with two sentences: one for the core purpose and one for parameter guidance. It's front-loaded with the main function and avoids unnecessary details. However, the second sentence could be slightly more structured (e.g., bullet points for clarity), preventing a perfect score.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (read-only, 2 parameters) and 100% schema coverage, the description is mostly adequate. However, with no output schema, it doesn't explain return values (e.g., product fields or error formats), and it lacks context on prerequisites or limitations, making it incomplete for optimal agent use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents both parameters. The description adds minimal value by mentioning the 'include' parameter and its purpose, but doesn't provide syntax or format details beyond what the schema already states (e.g., 'comma-separated list'). This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'retrieve a product from Paddle by its ID.' This is a specific verb+resource combination that distinguishes it from sibling tools like 'list_products' or 'create_product.' However, it doesn't explicitly differentiate from other 'get_' tools (e.g., 'get_price'), which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance by mentioning the 'include' parameter for related entities, suggesting when to use this tool for enriched data. However, it lacks explicit when-to-use vs. when-not-to-use statements or named alternatives (e.g., 'list_products' for multiple products). This makes it adequate but with gaps.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds useful context about retrieving configuration vs. run data, which helps the agent understand the scope. However, it doesn't disclose additional behavioral traits like error handling, authentication needs, or rate limits. With annotations covering safety, this adds moderate value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with zero waste—front-loaded with the core purpose and followed by clarifying context. It's appropriately sized for a simple retrieval tool, though it could be slightly more structured for optimal scanning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with one parameter and annotations covering safety, the description is mostly complete. However, without an output schema, it doesn't explain return values (e.g., what configuration data is included). Given the complexity is low, this is adequate but leaves a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with one parameter 'simulationId' fully documented. The description doesn't add any parameter-specific details beyond what the schema provides (e.g., format examples or constraints). Since the schema carries the full burden, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'retrieve a simulation from Paddle by its ID' (specific verb+resource). It distinguishes from sibling 'get_simulation_run' by clarifying it's for configuration vs. run. However, it doesn't explicitly differentiate from 'list_simulations' or 'create_simulation', which slightly limits full sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context by contrasting with 'simulation run' tools, suggesting this is for configuration retrieval. However, it doesn't provide explicit guidance on when to use this vs. 'list_simulations' or 'create_simulation', nor does it mention prerequisites or error conditions. The guidance is helpful but incomplete.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare readOnlyHint=true, so the agent knows this is a safe read operation. The description adds useful context about the 'include' parameter behavior (specifically that 'events' returns an array of event entities), which goes beyond what annotations provide. However, it doesn't describe other behavioral traits like error handling, rate limits, or authentication needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized with two sentences: one stating the purpose and one explaining the 'include' parameter. It's front-loaded with the core purpose. While efficient, the second sentence could be slightly more structured (e.g., using bullet points for clarity), but overall it avoids unnecessary verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (3 required parameters, no output schema), the description is adequate but has gaps. It covers the basic purpose and parameter usage but lacks information about return values, error cases, or how this fits into broader workflows with sibling tools. With annotations covering safety, it meets minimum viability but could be more comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description adds minimal value by briefly explaining the 'include' parameter's effect ('include related entities in the response') and listing the 'events' option, but this mostly repeats schema information. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'retrieve a simulation run from Paddle by its ID.' This is a specific verb ('retrieve') + resource ('simulation run') combination. However, it doesn't explicitly differentiate from sibling tools like 'get_simulation' or 'list_simulation_runs' beyond the obvious ID-based retrieval vs listing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides some implied usage guidance by mentioning the 'include' parameter for related entities, but it doesn't explicitly state when to use this tool versus alternatives like 'get_simulation' or 'list_simulation_runs.' There's no mention of prerequisites, error conditions, or specific contexts where this tool is preferred over others.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=true, confirming it's a safe read operation. The description adds valuable behavioral context beyond annotations: it details the response structure (status, payload, request.body, response.body, response.statusCode) and warns about tunnel/proxy services affecting interpretation, which aids in debugging and understanding output.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is moderately concise but could be more front-loaded. It starts with the purpose, then delves into detailed response fields and warnings. While informative, some sentences (e.g., about tunnel/proxy services) are lengthy and might bury key usage information, reducing efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (retrieving specific events with debugging details), no output schema, and rich annotations, the description is fairly complete. It explains the response structure and caveats, though it lacks explicit error handling or prerequisites, which could enhance completeness for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear parameter descriptions in the input schema. The tool description does not add any parameter-specific information beyond what the schema provides, such as format examples or usage tips, so it meets the baseline without extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'retrieve an event sent by a simulation run from Paddle by its ID.' It specifies the verb ('retrieve'), resource ('event'), and source ('Paddle'), but does not explicitly differentiate it from sibling tools like 'list_simulation_run_events' or 'get_simulation_run', which might cause confusion in selection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention sibling tools such as 'list_simulation_run_events' for multiple events or 'get_simulation_run' for run details, leaving the agent without context for tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate this is a non-destructive write operation (readOnlyHint: false, destructiveHint: false), which the description aligns with by stating it 'creates' something. The description adds useful context beyond annotations: it mentions that successful responses include the new entity and that events can be viewed via other tools. However, it doesn't cover potential side effects, error conditions, or rate limits, leaving some behavioral aspects unclear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and appropriately sized at three sentences. The first sentence states the purpose clearly, the second provides context about webhooks and simulations, and the third explains the response and related tools. There's minimal redundancy, and each sentence adds value, though it could be slightly more front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has one parameter with full schema coverage, non-destructive annotations, and no output schema, the description does a good job of providing context. It explains the tool's role in webhook testing, mentions the response format, and references related tools for event viewing. However, it could be more complete by explicitly stating prerequisites (e.g., simulation must exist) or error scenarios, which would help an agent use it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the single parameter 'simulationId' clearly documented as 'Paddle ID of the simulation to create a run for.' The description doesn't add any additional parameter details beyond what the schema provides, such as format examples or constraints. With high schema coverage, the baseline score of 3 is appropriate as the description doesn't compensate but doesn't need to.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool 'creates a new simulation run for a simulation in Paddle' and explains its purpose in the context of webhook testing. It distinguishes from sibling tools like 'create_simulation' (which creates the simulation itself) and 'get_simulation_run' (which retrieves one). However, it doesn't explicitly contrast with 'list_simulation_runs' or 'replay_simulation_run_event', leaving some sibling differentiation incomplete.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context by mentioning that simulation runs are used 'once the simulation has been configured' and for sending test webhook events. It suggests alternatives like 'list_simulations_events' or 'get_simulation_run' for viewing events, but doesn't explicitly state when to use this tool versus those alternatives or other siblings like 'create_simulation'. Guidance is present but not fully explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds valuable behavioral context beyond this: it discloses that the endpointSecretKey is returned for webhook verification and includes security warnings ('secure value,' 'never be shared,' 'store securely'). This enhances transparency about sensitive data handling, though it doesn't cover rate limits or error behaviors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded: the first sentence clearly states the purpose. The second sentence adds important security context without redundancy. Both sentences earn their place, though minor improvements in structure (e.g., bullet points for security warnings) could push it to a 5.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (single parameter, read-only operation) and rich annotations (readOnlyHint), the description is reasonably complete. It adds security warnings that aren't covered by annotations or schema. However, without an output schema, it doesn't detail the full response structure (e.g., other fields besides endpointSecretKey), leaving a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single parameter 'notificationSettingId,' which is fully documented in the schema. The description doesn't add any parameter-specific details beyond what the schema provides, such as format examples or constraints. With high schema coverage, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'retrieve a notification setting (notification destination) from Paddle by its ID.' This specifies the verb ('retrieve'), resource ('notification setting'), and scope ('by its ID'). However, it doesn't explicitly differentiate from sibling tools like 'list_notification_settings' or 'get_notification', which would require a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'list_notification_settings' for bulk retrieval or 'get_notification' for individual notifications, nor does it specify prerequisites or exclusions. This leaves the agent without contextual usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds valuable behavioral context beyond this: it explains the 'collectionMode' (automatic vs. manual) and detailed 'status' values (draft, ready, billed, etc.) with their meanings. This significantly enhances understanding of transaction states, though it doesn't cover rate limits or auth needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose, but it becomes verbose with detailed explanations of 'include' values, 'collectionMode,' and 'status.' While informative, some of this could be streamlined or moved to a separate section, as not every sentence directly aids tool selection in a concise manner.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (retrieval with optional includes and transaction states), the description provides substantial context: it explains parameters, related entities, and transaction behaviors. With annotations covering read-only safety and no output schema, it compensates well, though it could briefly mention response format or error handling for a higher score.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters. The description adds context for the 'include' parameter by listing and explaining each possible value (e.g., 'address' includes address entity if set). However, it doesn't provide additional syntax or format details beyond what the schema implies, meeting the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'retrieve a transaction from Paddle by its ID.' This is a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'get_transaction_invoice' or 'list_transactions,' which would be needed for a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by explaining the 'include' parameter for related entities, but doesn't explicitly state when to use this tool versus alternatives like 'list_transactions' or 'get_transaction_invoice.' It provides context on what the tool does but lacks explicit guidance on when to choose it over siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=true, which the description aligns with by describing a listing operation. The description adds valuable behavioral context beyond annotations: it explains pagination mechanics, default perPage usage, status interpretations, and caveats about tunnel/proxy services. This enhances transparency without contradicting annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose but becomes verbose with debugging details and caveats. While informative, some sentences could be more streamlined, and the structure mixes usage instructions with debugging advice, reducing overall efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, pagination, debugging needs) and lack of output schema, the description does a good job covering key aspects: purpose, usage, behavioral traits, and parameter hints. It compensates for missing output schema by explaining response fields, though it could be more organized.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 6 parameters. The description adds some semantic context, such as recommending maximum perPage and explaining pagination with 'after,' but does not provide significant additional meaning beyond the schema. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'list simulation run events in Paddle.' It specifies the resource (simulation run events) and verb (list), but does not explicitly differentiate it from sibling tools like 'list_simulation_runs' or 'get_simulation_run_event,' which reduces the score from a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance by detailing pagination, filtering, and sorting parameters, and it mentions debugging with status and response details. However, it lacks explicit when-to-use instructions compared to alternatives like 'get_simulation_run_event' or 'list_simulation_runs,' leaving some ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds valuable behavioral context beyond this: it explains the 90-day retention policy, error handling for non-retained notifications, and details about notification statuses and attributes (e.g., status, origin, timestamps). This enhances understanding of the tool's behavior without contradicting annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded with the core purpose, followed by important behavioral details. It avoids unnecessary fluff, but the detailed breakdown of notification attributes (status, origin, etc.) is somewhat lengthy for a retrieval tool, though informative. Most sentences earn their place by clarifying tool behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only tool with one parameter and no output schema, the description is quite complete. It covers the tool's purpose, retention policy, error conditions, and key return attributes. However, it doesn't specify the output format or structure, which could be helpful given the lack of an output schema, leaving a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the 'notificationId' parameter clearly documented. The description doesn't add any parameter-specific information beyond what the schema provides, such as format examples or constraints. Given the high schema coverage, a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'retrieve a notification from Paddle by its ID.' This specifies the verb ('retrieve') and resource ('notification'), making it unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'list_notifications' or 'get_notification_setting,' which would be needed for a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by mentioning that notifications older than 90 days aren't retained, suggesting this tool is for retrieving specific, recent notifications. However, it lacks explicit guidance on when to use this versus alternatives like 'list_notifications' or 'replay_notification,' and doesn't mention prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, but the description adds valuable behavioral context beyond that: it explains pagination mechanics ('use the after parameter with the last ID'), recommends a default perPage value (200), warns about secure handling of endpointSecretKey, and mentions sorting capabilities. This significantly enhances understanding of how the tool behaves in practice.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized (6 sentences) and front-loaded with the core purpose. Each sentence adds value: default behavior, filtering guidance, pagination mechanics, sorting, and security warning. There's minimal redundancy, though the security warning about endpointSecretKey could be slightly more concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with readOnlyHint annotation and no output schema, the description provides good completeness: it covers purpose, pagination behavior, filtering parameters, sorting, and security considerations. The main gap is lack of output format details (what fields are returned beyond endpointSecretKey), but given it's a list operation with good parameter coverage, this is acceptable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3. The description adds some semantic context: it explains that perPage defaults to 200 maximum, that active and trafficSource are filters 'as needed', and that after uses 'the last ID from previous results'. However, it doesn't provide significant additional meaning beyond what's already well-documented in the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'list notification settings in Paddle.' This is a specific verb+resource combination that distinguishes it from other list tools (like list_notifications, list_notification_logs). However, it doesn't explicitly differentiate from get_notification_setting (singular vs. plural), which would require a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance through parameter explanations (e.g., 'Filter notification settings by active and trafficSource as needed'), but doesn't explicitly state when to use this tool versus alternatives like get_notification_setting or list_notifications. It mentions pagination behavior but lacks explicit 'when-not' scenarios or sibling tool comparisons.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating it's a safe read operation. The description adds valuable behavioral context beyond this: it explains pagination behavior ('Results are paginated'), currency units ('Amounts are in the smallest currency unit'), and includes details on the 'include' parameter for related entities. This enhances understanding without contradicting annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded with the core purpose. It uses bullet points effectively for the 'include' parameter details, but some sentences could be more concise (e.g., the pagination explanation is slightly verbose). Overall, it's efficient with minimal waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (9 parameters, pagination, filtering) and the absence of an output schema, the description does a good job covering key aspects: purpose, pagination, filtering options, currency units, and related entities. However, it could improve by mentioning response format or error handling to be fully complete for a list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 9 parameters thoroughly. The description adds some semantic context, such as recommending a default for 'perPage' and explaining how 'after' works with pagination, but doesn't provide significant additional meaning beyond what the schema specifies. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'list prices in the account's catalog.' It specifies the verb ('list') and resource ('prices'), making it easy to understand. However, it doesn't explicitly differentiate from sibling tools like 'get_price' (singular) or 'preview_prices,' which might be used for different scenarios.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides some implied usage guidance, such as recommending to 'use the maximum perPage by default (200) to ensure comprehensive results' and explaining when to use the 'after' parameter for pagination. However, it lacks explicit when-to-use vs. alternatives (e.g., compared to 'get_price' for a single price or 'preview_prices' for estimates), and doesn't mention prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating this is a safe read operation. The description adds valuable behavioral context beyond annotations: it specifies pagination behavior ('Results are paginated'), recommends a default perPage value ('Use the maximum perPage by default (200)'), explains currency units ('Amounts are in the smallest currency unit'), and details the include parameter's effect. This enhances the agent's understanding without contradicting annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and appropriately sized, with clear sections on usage, filtering, pagination, sorting, currency, and includes. Each sentence adds value, such as the perPage recommendation and pagination instructions. It could be slightly more concise by avoiding repetition (e.g., 'Filter products by id, status, taxCategory, and type as needed' is somewhat redundant with schema details), but overall it's efficient and front-loaded with key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (8 parameters, no output schema), the description is reasonably complete. It covers key behavioral aspects like pagination, defaults, and response details (e.g., currency units, includes). With annotations handling the read-only safety, the description fills in necessary context without needing to explain return values. It could improve by mentioning error handling or rate limits, but it's sufficient for an agent to use the tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 8 parameters thoroughly. The description adds some semantic context: it mentions filtering by id, status, taxCategory, and type, and explains the include parameter with an example ('prices: An array of price entities tied to the product'). However, it doesn't provide significant additional meaning beyond what's in the schema, such as default values or edge cases, warranting a baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'list products in the account's catalog.' This is a specific verb ('list') and resource ('products'), but it doesn't distinguish this tool from other list_* siblings like list_prices or list_customers, which all follow the same pattern. The description could be more specific about what makes listing products unique.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance by mentioning filtering, pagination, sorting, and including related entities, but it doesn't explicitly state when to use this tool versus alternatives. For example, it doesn't compare list_products to get_product for retrieving a single product or explain its role relative to other list_* tools. The guidance is practical but lacks explicit context for tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, and the description adds valuable behavioral context beyond that: it specifies pagination behavior ('Results are paginated'), recommends default usage ('Use the maximum perPage by default'), and notes currency formatting ('Amounts are in the smallest currency unit'). This enhances understanding without contradicting annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized and front-loaded with the core purpose. Each sentence adds value: purpose, default behavior, filtering, pagination, sorting, and currency details. There's minimal waste, though it could be slightly more structured for clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity, 4 parameters with full schema coverage, readOnlyHint annotation, and no output schema, the description is reasonably complete. It covers key behavioral aspects like pagination and currency units, though it could benefit from more explicit sibling differentiation or error handling details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3. The description adds some semantic context (e.g., explaining pagination with 'after' and recommending 'perPage' defaults), but doesn't provide significant additional meaning beyond what the schema already documents for each parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'list reports in Paddle'. It specifies the verb ('list') and resource ('reports'), and while it doesn't explicitly differentiate from siblings like 'get_report' or 'create_report', the listing nature is evident. However, it doesn't fully distinguish from other list_* tools in the context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance through parameter explanations (e.g., 'Filter reports by status as needed'), but doesn't explicitly state when to use this tool versus alternatives like 'get_report' for individual reports or 'create_report' for new ones. It offers some context but lacks clear when/when-not directives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds valuable behavioral context beyond this: it specifies pagination behavior ('Results are paginated'), default usage ('Use the maximum perPage by default'), and what the 'include' parameter does (including related entities like events). This enhances the agent's understanding without contradicting annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and appropriately sized, with clear sentences that each serve a purpose: stating the tool's function, providing usage tips, and explaining the 'include' parameter. It's front-loaded with the core purpose and avoids redundancy, though it could be slightly more concise by integrating some tips into the parameter explanations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, pagination, filtering), the description is reasonably complete. It covers key behavioral aspects like pagination and default usage, and with annotations indicating read-only operation and no output schema, it doesn't need to explain return values. However, it could benefit from more explicit guidance on when to use versus sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 6 parameters. The description adds some semantic context, such as recommending default values for 'perPage' and explaining how 'after' works with pagination, but doesn't provide significant additional meaning beyond what's in the schema. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'list simulation runs in Paddle.' It specifies the resource (simulation runs) and verb (list), but doesn't explicitly differentiate from sibling tools like 'list_simulation_run_events' or 'get_simulation_run' beyond the resource name. The purpose is clear but lacks sibling distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance through parameter explanations (e.g., 'use the maximum perPage by default,' 'use the 'after' parameter... to get the next page'), but doesn't explicitly state when to use this tool versus alternatives like 'get_simulation_run' for single runs or 'list_simulation_run_events' for events. It offers practical tips but no direct comparative context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, but the description adds valuable behavioral context: it explains pagination mechanics ('after' parameter usage), recommends default perPage value (200), and clarifies that results are configurations rather than runs. This goes beyond what annotations alone provide without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded with the core purpose. Each sentence adds value: distinguishing simulations from runs, providing usage recommendations, and explaining pagination. While efficient, the recommendation about perPage could be slightly more concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the read-only nature (annotations), comprehensive parameter documentation (schema), and lack of output schema, the description provides good contextual completeness. It covers key behavioral aspects like pagination and filtering scope, though it doesn't describe the return format or error conditions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the input schema already documents all 6 parameters thoroughly. The description adds minimal parameter-specific semantics beyond the schema, mainly reinforcing filtering capabilities and pagination behavior. This meets the baseline expectation when schema coverage is complete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool 'list simulations in Paddle' and distinguishes simulations from simulation runs, providing specific context about what type of resource is being listed. However, it doesn't explicitly differentiate from sibling list tools like 'list_simulation_runs' beyond the general distinction mentioned.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by mentioning filtering parameters and pagination, but doesn't provide explicit guidance on when to use this tool versus alternatives like 'get_simulation' or 'list_simulation_runs'. It offers some operational context but lacks clear when/when-not directives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, and the description adds valuable behavioral context beyond that: it specifies pagination behavior ('Results are paginated'), recommends a default perPage value ('Use the maximum perPage by default (200)'), and notes currency formatting ('Amounts are in the smallest currency unit'). This enhances understanding without contradicting annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized with 5 sentences, each adding value: purpose, default usage, filtering, pagination, sorting, and currency details. It's front-loaded with the core purpose and key usage tip. Minor redundancy exists in listing filter fields that are already in the schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (10 parameters, no output schema), the description is fairly complete: it covers purpose, key behavioral traits (pagination, defaults, currency), and usage hints. With annotations covering read-only safety, it doesn't need to explain return values deeply, though output format details could be more explicit.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 10 parameters thoroughly. The description adds minimal param semantics by listing filterable fields and mentioning pagination with 'after', but doesn't provide significant additional meaning beyond what's in the schema. Baseline 3 is appropriate given high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose as 'list subscriptions in Paddle,' which is a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'get_subscription' or other list tools, though the context suggests it's for bulk retrieval with filtering.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance by mentioning filtering, pagination, and sorting, but doesn't explicitly state when to use this tool versus alternatives like 'get_subscription' for single items or other list tools for different resources. No exclusions or clear alternatives are named.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=true, and the description aligns with this by describing a listing operation without contradictions. It adds valuable behavioral context beyond annotations, such as recommending a default perPage value (200), explaining pagination mechanics with 'after' parameter, and noting that results are paginated—details not covered by annotations alone.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and concise, with four sentences that each serve a clear purpose: stating the tool's purpose, recommending a default, explaining filtering, and detailing pagination and sorting. No wasted words, and it's front-loaded with essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (7 parameters, pagination) and lack of output schema, the description does a good job covering key aspects like default usage, filtering, and pagination. However, it doesn't detail the response format or error handling, which could be helpful for an agent. With annotations providing safety info, it's mostly complete but has minor gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds some semantic context by explaining how to use 'after' for pagination and 'search' for fuzzy matching, but this mostly reinforces rather than extends the schema. Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose as 'list addresses for a customer in Paddle,' which is a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'get_address' (singular) or other list tools, though the context of listing addresses for a specific customer is reasonably distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance by mentioning filtering, pagination, and sorting, but it doesn't explicitly state when to use this tool versus alternatives like 'get_address' or other list tools. No clear exclusions or prerequisites are mentioned, leaving some ambiguity for the agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, which the description aligns with by describing a listing operation. The description adds valuable behavioral context beyond annotations: pagination mechanics ('use the 'after' parameter with the last ID'), default behavior ('use the maximum perPage by default'), and filtering capabilities. It doesn't mention rate limits or authentication needs, but with annotations covering safety, this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently structured in 4 sentences, each serving a distinct purpose: stating the tool's function, recommending a default, explaining filtering options, and detailing pagination/sorting. There's no redundant information, and key guidance is front-loaded. Every sentence earns its place by adding practical value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (7 parameters, pagination, filtering), the description provides good context. With annotations covering read-only safety and 100% schema coverage for parameters, the description adds necessary behavioral details like pagination mechanics and default usage. The lack of an output schema is a minor gap, but the description compensates by explaining result structure indirectly through parameter guidance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 7 parameters. The description adds some semantic context: it explains the purpose of filtering parameters ('filter customers by email, id, search...'), recommends default values ('use the maximum perPage by default'), and clarifies pagination usage. However, it doesn't provide significant additional meaning beyond what's already in the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'list customers in Paddle' with specific filtering capabilities. It distinguishes itself from sibling tools like 'get_customer' (singular retrieval) by focusing on listing multiple customers with filtering options. However, it doesn't explicitly contrast with other list_* tools (e.g., list_addresses, list_subscriptions) which might have similar patterns.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implicit usage guidance through parameter explanations (e.g., 'use the maximum perPage by default', 'use the 'after' parameter for pagination'). However, it doesn't explicitly state when to use this tool versus alternatives like 'get_customer' for single customer retrieval or 'search' for fuzzy matching. The guidance is practical but not comparative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds valuable behavioral context beyond this: it specifies pagination behavior ('Results are paginated'), recommends a default perPage value ('Use the maximum perPage by default (200)'), and explains how to navigate pages ('use the 'after' parameter with the last ID'). This enhances the agent's understanding of how to use the tool effectively, though it doesn't cover rate limits or authentication needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose, followed by specific usage tips in a bullet-like structure. Each sentence adds practical value: default settings, filtering, pagination mechanics, and sorting. There is no wasted text, and the information is efficiently organized for quick comprehension by an agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (a list operation with pagination and filtering), the description is reasonably complete. It covers key behavioral aspects like pagination and default usage, and annotations handle the safety profile. However, without an output schema, it doesn't describe the return format (e.g., structure of discount group objects), which is a minor gap for a list tool. Sibling tools provide context, but the description doesn't leverage this for differentiation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds some semantic context: it emphasizes using the maximum perPage for comprehensive results, clarifies that 'after' uses the last ID for pagination, and mentions filtering by ID and sorting with orderBy. However, this mostly reinforces rather than significantly extends the schema information, meeting the baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'list discount groups in the account's catalog.' This specifies the verb ('list') and resource ('discount groups'), and the context ('account's catalog') provides helpful scope. However, it doesn't explicitly differentiate from sibling tools like 'get_discount_group' (singular) or 'list_discounts', which would be needed for a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance by mentioning pagination, filtering by ID, and sorting, suggesting this tool is for browsing or searching discount groups. However, it lacks explicit when-to-use directives, such as contrasting with 'get_discount_group' for single entities or explaining scenarios where listing is preferred over getting specific IDs. No alternatives or exclusions are named.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating safe read operations. The description adds valuable behavioral context beyond annotations: pagination mechanics ('use the after parameter with the last ID'), default behavior ('use maximum perPage by default'), data format ('amounts are in smallest currency unit'), and filtering capabilities. It doesn't mention rate limits or authentication needs, but adds significant operational details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently structured with 5 sentences, each providing distinct value: purpose statement, default usage, filtering options, pagination instructions, and data format note. It's front-loaded with the core purpose and avoids redundancy. Every sentence earns its place without wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (7 parameters, pagination, filtering), annotations cover safety (readOnlyHint), and schema provides full parameter documentation. The description adds important operational context: pagination mechanics, default behavior, and data format. Without an output schema, it doesn't describe return values, but for a list operation with good schema coverage, this is reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are well-documented in the schema. The description adds some semantic context: it mentions filtering by 'code, id, status, and mode' (matching schema parameters) and explains pagination with 'after' and sorting with 'orderBy'. However, it doesn't provide additional meaning beyond what the schema already describes, such as parameter interactions or edge cases.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'list discounts in the account's catalog.' It specifies the verb ('list') and resource ('discounts'), but doesn't explicitly differentiate from sibling tools like 'get_discount' or 'list_discount_groups' beyond the name. The purpose is clear but lacks sibling comparison context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance: 'Use the maximum perPage by default (200) to ensure comprehensive results' suggests a best practice, and mentions filtering, pagination, and sorting. However, it doesn't explicitly state when to use this tool versus alternatives like 'get_discount' (for single discounts) or 'list_discount_groups', nor does it mention prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds valuable behavioral context beyond this: it explains what the 'include' parameter does (e.g., 'next_transaction' includes prorated charges, 'recurring_transaction_details' shows expected billing), which helps the agent understand response behavior. No contradictions with annotations are present.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized and front-loaded: the first sentence states the core purpose, followed by specific usage details for the 'include' parameter. Every sentence adds value without redundancy, making it efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (2 parameters, read-only operation), the description is fairly complete. It covers purpose, parameter usage, and behavioral details. However, with no output schema, it doesn't describe the return format (e.g., subscription object structure), leaving a minor gap. Annotations help by indicating safety.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents parameters. The description adds semantic meaning by explaining the purpose and options of the 'include' parameter (e.g., 'next_transaction' includes unbilled charges), which goes beyond the schema's enum list. This compensates well, though it doesn't detail 'subscriptionId' further.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'retrieve a subscription from Paddle by its ID.' This is a specific verb ('retrieve') and resource ('subscription'), but it doesn't explicitly differentiate from sibling tools like 'list_subscriptions' or 'get_customer' (which might also retrieve subscription data). The description is clear but lacks sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance by explaining the 'include' parameter for related entities, suggesting when to use this tool for detailed subscription data. However, it doesn't explicitly state when to choose this over alternatives like 'list_subscriptions' for bulk data or other 'get_' tools for different resources. The guidance is contextual but not comprehensive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds valuable behavioral context beyond the readOnlyHint annotation: it explains that events are created for all notable actions regardless of notification settings, that some actions create multiple events, and provides specific pagination guidance (maximum 200 per page, use 'after' for next page). This goes well beyond what the annotation alone provides about read-only access.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and appropriately sized. It starts with the core purpose, provides context about events, gives important behavioral details, and ends with specific usage instructions. Each sentence adds value, though the middle paragraph about event creation could be slightly more concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with readOnlyHint annotation and good schema coverage, the description provides comprehensive context. It explains what events are, how they're created, pagination behavior, and usage patterns. The main gap is lack of output format details (no output schema exists), but otherwise it's quite complete for this type of tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3. The description adds some value by explaining practical usage of parameters (use 'after' with last ID for pagination, use orderBy for sorting, perPage maximum is 200), but doesn't provide additional semantic meaning beyond what's already in the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'list events in Paddle' with additional context about what events are ('notable occurrences' with entity creation). It distinguishes from siblings by focusing specifically on events rather than other resources like addresses, customers, or transactions. However, it doesn't explicitly differentiate from other list_* tools beyond the resource type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage guidance for pagination ('use the after parameter'), sorting ('use the orderBy parameter'), and default behavior ('use maximum perPage by default'). It explains when to use pagination parameters but doesn't explicitly state when to choose this tool over alternatives or mention any prerequisites for using it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, suggesting this is a non-destructive operation that may involve writes (e.g., generating preview data). The description adds value by clarifying that changes are not applied ('without applying those changes') and detailing the response structure (e.g., updateSummary, transaction details). However, it doesn't disclose behavioral traits like rate limits, authentication needs, or error handling, which are not covered by annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized and front-loaded, starting with the core purpose. Each sentence adds value: the first states the action, the second provides usage guidance, and the last two explain response details. There's no wasted text, but it could be slightly more structured (e.g., bullet points for response fields) without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (14 parameters, nested objects) and lack of output schema, the description does well by explaining key response components (updateSummary, transaction details). However, it doesn't cover all contextual aspects like error cases, prerequisites (e.g., required permissions), or how it integrates with sibling tools (e.g., 'update_subscription' not listed). With annotations providing safety hints, it's mostly complete but has minor gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, thoroughly documenting all 14 parameters. The description adds minimal parameter semantics beyond the schema, only implying that parameters relate to subscription updates (e.g., 'items, billing periods, and anything affecting proration'). With high schema coverage, the baseline is 3, as the description doesn't significantly enhance parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'preview an update for a subscription without applying those changes.' It specifies the verb ('preview') and resource ('subscription update'), distinguishing it from tools like 'update_subscription' (though not listed as a sibling). However, it doesn't explicitly differentiate from sibling tools like 'preview_subscription_charge' or 'preview_prices', which are also preview tools in the same domain, leaving some ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidelines: 'It's best practice to preview every time before updating the subscription to confirm the changes are as expected, especially when making updates to items, billing periods, and anything affecting proration.' This clearly indicates when to use this tool (before applying updates) and highlights critical scenarios, though it doesn't name specific alternative tools like 'update_subscription' (not in siblings).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate this is a mutation tool (readOnlyHint: false), and the description correctly describes it as a creation operation. It adds valuable behavioral context beyond annotations by explaining the business impact (addresses needed for purchases), requirements (country mandatory, postalCode sometimes required), and what happens on success (response includes new address entity). No contradiction with annotations exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately front-loaded with purpose and context, but includes generic agent guidance ('Ensure you have all the information...', 'Don't fabricate...') that doesn't add tool-specific value. These sentences could be removed without losing essential tool understanding, making the description somewhat verbose for its core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no output schema, the description provides good context about what the tool does, when to use it, requirements, and success behavior. It covers the essential aspects given the annotations (which indicate it's non-destructive) and parameter coverage. The main gap is lack of explicit error handling or rate limit information, but overall it's fairly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the schema already documents all 9 parameters thoroughly. The description adds some semantic context about country and postalCode requirements (e.g., 'postalCode is required when creating addresses for some countries'), but doesn't provide significant additional meaning beyond what's in the schema descriptions. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with specific verb ('create') and resource ('address for a customer in Paddle'), and distinguishes it from siblings by explaining that addresses are needed for purchases and transactions. It provides context about address entities holding billing information, which differentiates it from other 'create_' tools like create_customer or create_transaction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context about when addresses are needed ('Customers must have an address to make a purchase', 'transaction can't go past draft until an address is added'), and mentions country requirements for tax/fraud purposes. However, it doesn't explicitly state when NOT to use this tool or name specific alternatives among the sibling tools, such as when to use get_address or list_addresses instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, covering basic safety. The description adds valuable behavioral context beyond annotations: it explains that image URLs must be publicly accessible HTTPS, Paddle doesn't upload images to a CDN, tax category selection impacts tax rates, and the response includes a copy of the new product entity. It also warns about tax category review for the 'standard' category. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the tool's purpose, but it includes extensive tax category explanations and general usage warnings that could be streamlined. While informative, some sentences (e.g., the detailed tax category list) are lengthy, and the final paragraph about not fabricating details is generic rather than tool-specific, reducing efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, 2 required, nested objects) and lack of output schema, the description is mostly complete. It covers key behavioral aspects like image hosting, tax category implications, and response format. However, it could better address error handling or specific constraints for parameters like customData to fully compensate for the missing output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds meaning for taxCategory by listing and explaining each enum value in detail, and for imageUrl by specifying HTTPS requirements and image ratio recommendations. However, it does not provide additional context for other parameters like name, description, type, or customData beyond what the schema offers.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'create a new product in Paddle' with the specific verb 'create' and resource 'product'. It distinguishes from siblings by focusing on product creation rather than other entities like addresses, adjustments, or customers, and explains that 'Product entities describe the items that customers can purchase'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool: for creating products that work with prices and require tax categories. It includes guidance on prerequisites ('Ensure you have all the information needed before making the call') and post-creation steps ('Once a product has been created, relate it to a price'). However, it does not explicitly mention when not to use it or name specific alternatives among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds valuable behavioral context beyond this: it specifies default pagination behavior ('maximum perPage by default (200)'), explains pagination mechanics ('use the 'after' parameter with the last ID'), and details filtering options (id, search, status). This enriches the agent's understanding of how the tool behaves in practice, though it doesn't cover rate limits or auth needs. The description complements annotations well without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently structured with four sentences, each serving a distinct purpose: stating the tool's purpose, recommending a default, detailing filtering options, and explaining pagination and sorting. There is no wasted text, and information is front-loaded with the core function. This makes it easy for an agent to parse and apply the guidance quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (7 parameters, pagination, filtering) and the presence of annotations (readOnlyHint) but no output schema, the description does a good job of covering key aspects: purpose, usage, and behavioral traits. It explains pagination, filtering, and sorting, which are critical for effective use. However, it doesn't describe the return format or error handling, leaving some gaps. With annotations providing safety context, it's largely complete but not exhaustive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, meaning all parameters are documented in the schema. The description adds some semantic context: it clarifies that 'search' performs 'fuzzy search on the business's name or tax or VAT number' (vs. schema's 'exact match'), and it provides usage tips for 'after' and 'orderBy.' However, it doesn't significantly enhance meaning beyond the schema, such as explaining parameter interactions or edge cases. Baseline 3 is appropriate given high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'list businesses for a customer in Paddle.' It specifies the verb ('list'), resource ('businesses'), and scope ('for a customer'), which is specific and actionable. However, it doesn't explicitly differentiate from sibling tools like 'list_customers' or 'get_business,' though the scope implies it's for businesses under a specific customer. This clarity earns a 4, as it's clear but lacks explicit sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use this tool: for listing businesses with filtering, pagination, and sorting capabilities. It implicitly suggests usage for comprehensive results by recommending 'maximum perPage by default (200).' However, it doesn't explicitly state when not to use it or name alternatives (e.g., 'get_business' for single businesses), which prevents a perfect score. The guidance is strong but not exhaustive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds valuable behavioral context beyond this: it explains pagination mechanics, retention limits (90 days), and detailed status/enum explanations (e.g., status values like 'needsRetry', origin types). This enriches the agent's understanding of how the tool behaves in practice, though it could mention rate limits or authentication needs more explicitly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose but becomes verbose with detailed explanations of statuses and other attributes that might be better suited for an output schema. While informative, some sentences (e.g., the lengthy status breakdown) could be trimmed for conciseness without losing essential guidance, making it slightly over-specified in parts.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (9 parameters, no output schema), the description is quite complete. It covers purpose, usage, behavioral details like pagination and retention, and parameter guidance. However, without an output schema, it partially compensates by explaining returned attributes (e.g., status, origin), but could more explicitly structure this as return value documentation to be fully comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 9 parameters thoroughly. The description adds some semantic context by explaining how to use parameters (e.g., 'use the maximum perPage by default', 'filter notifications by...', 'use the 'after' parameter with the last ID'), but this mostly reinforces rather than significantly extends the schema. Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb ('list') and resource ('notifications in Paddle'), and distinguishes it from siblings by focusing on notification entities rather than other resources like addresses, customers, or transactions. It explains what notifications are and how they relate to events, providing essential context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use this tool by explaining the nature of notifications and their retention policy (older than 90 days aren't returned). It suggests default usage ('Use the maximum perPage by default') and lists filtering parameters. However, it does not explicitly state when to use alternatives like 'get_notification' or 'list_notification_logs', missing explicit sibling differentiation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate this is a non-destructive write operation (readOnlyHint: false, destructiveHint: false). The description adds valuable behavioral context beyond annotations: it explains the response format (includes the new token entity), safety considerations (can be exposed client-side safely), and environment-specific behaviors (test_ vs. live_ token prefixes and their implications).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded with the core purpose, followed by usage context and behavioral details. Most sentences add value, though the explanation of token prefixes could be slightly more concise. Overall, it efficiently conveys necessary information without unnecessary verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a creation tool with no output schema, the description does a good job covering key aspects: purpose, usage, parameters (via schema), behavioral traits, and response details. It could be more complete by explicitly mentioning error conditions or prerequisites, but it adequately addresses the tool's complexity given the available annotations and schema coverage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, so the schema already documents both parameters (name and description). The description adds some semantic context by advising to 'provide a descriptive name to help identify its purpose', but does not provide additional details beyond what the schema descriptions offer.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('create a new client-side token in Paddle'), identifies the resource ('client-side token'), and distinguishes it from siblings by explaining its unique purpose for authenticating with Paddle.js. This goes beyond just restating the tool name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool ('needed to authenticate with Paddle.js', 'usually created for each application or environment'), but does not explicitly mention when not to use it or name specific alternatives among the sibling tools (e.g., get_client_side_token or list_client_side_tokens).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds significant behavioral context beyond annotations. It explains collection modes (manual vs. automatic), status implications (billed transactions become legal records), subscription creation for recurring items, and response inclusion options. Annotations only indicate it's not read-only or destructive, so the description provides essential operational details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is excessively long (over 600 words) and poorly structured. It mixes usage warnings, parameter explanations, and behavioral details without clear organization. While informative, it lacks front-loading and contains repetitive elements, making it difficult for an agent to quickly extract key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (13 parameters, nested objects, no output schema), the description provides substantial context about behavior, usage scenarios, and implications. It covers collection modes, status handling, subscription creation, and response inclusions. However, it could better summarize the tool's overall workflow and error conditions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds some semantic context for parameters like 'collectionMode', 'status', and 'include', explaining their effects and usage scenarios, but doesn't provide extensive additional parameter details beyond what the schema already documents.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'create a new transaction in Paddle.' It specifies the verb ('create') and resource ('transaction'), and distinguishes it from sibling tools like 'preview_transaction_create' and 'get_transaction' by emphasizing the creation action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool: 'Don't use this tool without checking with the user first' and 'Consider using the preview_transaction_create tool to preview and confirm the transaction before creating it.' It also mentions alternatives like using checkouts for automatically-collected transactions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=true, which the description aligns with by using 'retrieve' (implying a read operation). The description adds valuable context beyond annotations: it clarifies what type of payment methods are retrieved (saved at checkout, not for subscriptions) and provides guidance on alternative approaches for subscription-related payment methods. However, it doesn't mention potential errors, rate limits, or authentication requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and appropriately sized. The first sentence clearly states the purpose, followed by explanatory context about what these payment methods are and aren't, and ends with guidance on alternatives. Every sentence adds value, though the second sentence could be slightly more concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (retrieving a specific resource), 100% schema coverage, and readOnlyHint annotation, the description is largely complete. It clarifies the tool's scope and provides usage guidance. The main gap is the lack of output schema, but the description compensates somewhat by explaining what type of data is retrieved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both parameters (customerId and paymentMethodId) clearly documented in the schema. The description adds minimal semantic context beyond the schema, only implying these IDs are Paddle-specific. Since the schema does the heavy lifting, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'retrieve a payment method for a customer from Paddle using its ID and related customer ID.' It specifies the verb ('retrieve'), resource ('payment method'), and distinguishes it from sibling tools like 'list_saved_payment_methods' (which lists multiple) and 'list_transactions' (which shows recently used methods).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool vs. alternatives. It states these are 'payment methods saved by the customer at checkout to be presented for future purchases' and clarifies they 'aren't payment methods stored for transactions related to a recurring subscription.' It also explicitly names an alternative tool ('list_transactions') for viewing recently used payment methods.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=true, which the description aligns with by describing a listing operation. The description adds valuable behavioral context beyond annotations: pagination mechanics ('after' parameter usage), default pagination behavior (maximum perPage=50), currency unit details (amounts in smallest unit like cents), and detailed explanations of adjustment actions including which are manual vs. automatic. This significantly enhances the agent's understanding of how the tool behaves.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized and front-loaded with the core purpose and key usage guidelines. The detailed action explanations are necessary for behavioral transparency. While slightly lengthy, every section adds value and there's no redundant information. It could be slightly more concise but remains well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (9 parameters, no output schema) and rich annotations (readOnlyHint), the description provides comprehensive context. It covers purpose, usage, behavioral details (pagination, currency, action types), and parameter guidance. The main gap is the lack of output schema, but the description compensates well with behavioral explanations. For a listing tool with good annotations, this is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 9 parameters thoroughly. The description adds some context about parameter usage (e.g., 'after' parameter with last ID for pagination, filtering 'as needed'), but doesn't provide significant additional semantic meaning beyond what's in the schema. This meets the baseline expectation when schema coverage is complete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with a clear verb+resource statement: 'This tool will list adjustments in Paddle.' It distinguishes from siblings like 'get_adjustment_credit_note' (specific adjustment detail) and 'create_adjustment' (write operation), establishing its role as a comprehensive listing tool for adjustments.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool: for listing adjustments with filtering, pagination, and sorting capabilities. It doesn't explicitly state when NOT to use it or name alternatives, but the context is sufficient for an agent to infer this is the primary listing tool for adjustments among the siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating this is a safe read operation. The description adds valuable behavioral context beyond annotations: it explains pagination behavior ('Results are paginated'), safety of exposing tokens client-side, environment distinctions (test_ vs. live_ tokens), and token status meanings (active vs. revoked). This enriches the agent's understanding without contradicting annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded with the core purpose. It uses bullet points for token prefixes and statuses, which aids readability. However, some sentences could be more concise (e.g., 'The returned token field is the client-side token that needs to be provided when initializing Paddle.js' is slightly redundant). Overall, it efficiently conveys necessary information without excessive verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (list operation with pagination and filtering), annotations (readOnlyHint), and schema coverage (100%), the description is quite complete. It explains key behaviors like pagination, token safety, and status meanings. The lack of an output schema is compensated by describing the returned token field and its implications. Minor gaps include not detailing the output structure beyond the token field.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters (after, orderBy, perPage, status). The description adds some semantic context: it recommends using maximum perPage (200), explains how to use 'after' for pagination, and clarifies status filtering. However, it does not provide significant additional meaning beyond what the schema offers, such as default values or usage examples for orderBy.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'list client-side tokens in Paddle.' It specifies the resource (client-side tokens) and the action (list), and distinguishes it from sibling tools like 'get_client_side_token' (singular) and 'create_client_side_token' (creation). The description also explains what client-side tokens are used for (authenticating with Paddle.js), adding context beyond just the verb.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage guidance: 'Use the maximum perPage by default (200) to ensure comprehensive results' and 'Filter client-side tokens by status as needed.' It also explains pagination with the 'after' parameter. However, it does not explicitly state when to use this tool versus alternatives like 'get_client_side_token' (for a single token) or other list tools, though the context of listing tokens is implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, and the description aligns with this by describing a listing/log examination function. The description adds valuable behavioral context beyond annotations: explains pagination behavior, default usage of maximum perPage (200), and what details to check for debugging. It doesn't mention rate limits or authentication requirements, but adds useful operational context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and appropriately sized. It starts with the core purpose, provides context about what notification logs are, then gives usage guidance and debugging details. Some sentences could be more concise (e.g., the debugging details section is quite detailed), but overall it's efficient and front-loaded with essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only listing tool with 3 parameters and no output schema, the description provides good completeness. It explains what the tool returns (notification logs with specific fields), how to use pagination, and debugging context. The main gap is lack of output format details since there's no output schema, but the description compensates reasonably by listing key response fields to examine.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are well-documented in the schema. The description adds some semantic context: explains that 'after' should use the last ID from previous results for pagination, and that perPage maximum is 200. However, it doesn't provide additional meaning for notificationId beyond what's in the schema. Baseline 3 is appropriate given high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'list notification logs in Paddle' with specific context about what notification logs are (delivery attempt records). It distinguishes from sibling tools like 'list_notifications' by focusing specifically on logs rather than notifications themselves, and from 'replay_notification' by being read-only versus action-oriented.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool: to examine delivery attempts and debug issues. It mentions using maximum perPage by default and pagination with 'after' parameter. However, it doesn't explicitly state when NOT to use it or name specific alternatives among siblings, though the context implies it's for logs rather than notifications.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, but the description adds valuable behavioral context beyond this. It explains that providing location information enables tax calculation and price localization, describes how formatted totals are returned in the response, and specifies that successful responses include a 'details' object with totals. No contradiction with annotations exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and appropriately sized. It starts with the core purpose, then provides usage guidelines, parameter context, and response details. Most sentences add value, though some information (like the formatted totals explanation) could be slightly more concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (8 parameters, tax calculation logic) and lack of output schema, the description does a good job explaining key aspects: purpose, when to use alternatives, location parameter significance, and response format. However, it could better explain the relationship between items and prices or provide more detail about error cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 8 parameters thoroughly. The description adds some semantic context by explaining the purpose of location parameters (customer_ip_address, address, customerId/addressId/businessId) for tax calculation and localization, but doesn't provide significant additional meaning beyond what's in the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'preview price calculations for one or more prices.' It specifies the verb ('preview') and resource ('price calculations'), but doesn't explicitly differentiate from sibling tools like 'preview_subscription_charge' or 'preview_transaction_create' beyond mentioning the latter as an alternative for advanced calculations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool versus alternatives: 'Consider using the preview_transaction_create tool for more advanced and accurate pricing calculations or for all manually-collected invoiced transactions.' It also details context for location-based calculations, specifying three scenarios (customer_ip_address, address, customerId/addressId/businessId) and their typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds valuable behavioral context beyond the annotations. While annotations indicate it's not read-only and not destructive, the description explains: 'Billed transactions are considered financial records for tax and legal purposes, so they can't be changed. Adjustments record actions that impact revenue for a transaction after it's been billed.' It also details approval workflows: 'Most refunds for live accounts are created with the status of pending_approval until reviewed by Paddle, but some are automatically approved. For sandbox accounts, Paddle automatically approves refunds every ten minutes.'
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is comprehensive but somewhat verbose at 18 sentences. While most content is valuable, some sections could be more concise (e.g., the repeated warnings about checking with users). The structure is logical but not optimally front-loaded, with usage guidelines appearing after technical explanations rather than immediately after the purpose statement.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of financial adjustments and the absence of an output schema, the description provides substantial context about the tool's behavior, constraints, and business logic. It explains the distinction between refunds and credits, approval workflows, tax modes, and transaction state requirements. However, it could provide more detail about the response structure since there's no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3. The description adds some semantic context about parameters (e.g., explaining when to use 'refund' vs 'credit', 'full' vs 'partial' adjustments, and 'external' vs 'internal' tax modes), but most parameter details are already well-covered in the schema descriptions. The description doesn't add significant new parameter information beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'create an adjustment to refund or credit all or part of a transaction and its items.' It distinguishes this from sibling tools like 'create_transaction' by focusing specifically on post-billing adjustments rather than initial transactions, and explains why adjustments are needed (billed transactions can't be changed).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool vs alternatives: 'Don't use this tool without checking with the user first. Avoid using before gaining explicit approval.' It also specifies prerequisites: 'Ensure you have all the information needed before making the call' and warns against creating adjustments when 'a transaction that has a refund that's pending approval.'
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds valuable behavioral context beyond the readOnlyHint annotation. It explains that the link expires after an hour, which is crucial operational information not captured in annotations. It also clarifies that credit note PDFs are created for refunds and credits as adjustment records, providing context about when these documents exist. No contradiction with the readOnlyHint annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is perfectly structured and concise with three focused sentences. The first states the core purpose, the second provides context about when credit notes are created, and the third delivers crucial behavioral information about link expiration. Every sentence earns its place with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only retrieval tool with good annotations and complete schema coverage, the description provides excellent context about the tool's purpose, usage timing, and link expiration behavior. The main gap is the lack of output schema, so the description doesn't specify the exact format of the returned link, but this is mitigated by the clear behavioral context provided.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the schema already documents both parameters thoroughly. The description doesn't add any additional parameter semantics beyond what's in the schema descriptions. The baseline score of 3 is appropriate since the schema does the heavy lifting for parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with specific verbs ('retrieve a link to a credit note PDF') and resource ('for an adjustment from Paddle'). It distinguishes itself from siblings by focusing on credit note PDF retrieval rather than general adjustment data (like 'get_adjustment' or 'list_adjustments'), and explains the context of credit notes being created for refunds and credits.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool ('Return this if record is needed to be given to the customer immediately after the adjustment is created'), which helps differentiate it from other adjustment-related tools. However, it doesn't explicitly mention when NOT to use it or name specific alternatives among the sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, indicating a safe read operation. The description adds valuable behavioral context beyond this: it explains the token's purpose (initializing Paddle.js), safety considerations (can be exposed client-side), environment distinctions (test_ vs. live_ prefixes), and token statuses (active/revoked). This enhances the agent's understanding without contradicting annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded, starting with the core purpose, followed by usage context and behavioral details. Each sentence adds value: the first states the action, the second explains the token's use, and the subsequent ones cover safety, environment prefixes, and statuses—with zero wasted content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, read-only operation) and lack of an output schema, the description provides comprehensive context: it covers purpose, usage, behavioral traits, and output interpretation (token prefixes and statuses). However, it does not explicitly describe the return format or error handling, leaving minor gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the parameter 'clientTokenId' clearly documented as 'Paddle ID of the client-side token.' The description does not add further parameter details beyond what the schema provides, so it meets the baseline of 3 for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('retrieve a client-side token') and resource ('from Paddle by its ID'), distinguishing it from sibling tools like 'list_client_side_tokens' (which lists multiple tokens) and 'create_client_side_token' (which creates new tokens). It precisely defines what the tool does without ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use this tool (to retrieve a specific token by ID) and implicitly distinguishes it from 'list_client_side_tokens' for bulk retrieval. However, it lacks explicit guidance on when NOT to use it or alternatives for related operations like token creation or listing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations provide readOnlyHint=true, but the description adds valuable behavioral context beyond this: it explains that credit balances are automatically created by Paddle for transaction credits (not promotional), describes the three balance totals (available, reserved, used) and their lifecycle, states that response isn't paginated, and clarifies that empty arrays are returned for no balances. This significantly enhances understanding of the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized and front-loaded with the core purpose. While comprehensive, some sentences could be more concise (e.g., the reserved total explanation is detailed but necessary). The structure flows logically from purpose to credit explanation to balance details to usage notes.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of credit balances and the absence of an output schema, the description provides excellent contextual completeness. It thoroughly explains what credit balances are, how they're created, the three balance types and their relationships, currency restrictions, how to affect balances (via create_adjustment), filtering, amount format, and response behavior. This compensates well for the lack of output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the schema already documents both parameters well. The description adds minimal parameter-specific information beyond the schema - it mentions filtering by currencyCode and that amounts are in smallest currency units, but doesn't provide additional semantic context about the customerId parameter or currencyCode usage beyond what's in the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('list credit balances in each currency for a customer'), identifies the resource ('credit balances'), and distinguishes it from sibling tools by focusing on credit balances rather than other entities like addresses, adjustments, or transactions. The opening sentence provides immediate clarity about the tool's function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context about when to use this tool (to list credit balances created by Paddle for transaction credits) and explicitly mentions an alternative tool ('create_adjustment') for adding to credit balances. However, it doesn't explicitly state when NOT to use this tool versus other list_* siblings, though the credit balance focus is distinct enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, but the description adds valuable behavioral context beyond this: it explains that the tool returns authenticated links that automatically sign in the customer, specifies that sessions are temporary and shouldn't be cached, and details what happens for paused/canceled subscriptions. This enhances understanding of the tool's behavior without contradicting annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with clear sections: purpose, portal capabilities, usage warnings, authorization notes, URL details, and behavioral notes. While comprehensive, some sentences could be more concise (e.g., the list of portal capabilities is lengthy but informative). Overall, it's front-loaded with key information and avoids redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (creating authenticated sessions with security implications), the description provides substantial context: it covers purpose, usage guidelines, behavioral traits, and output details (URLs and session entity). However, without an output schema, it doesn't fully document the response structure (e.g., exact fields in the 'new customer portal session entity'). This minor gap prevents a perfect score.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear documentation for both parameters (customerId and subscriptionIds). The description adds some semantic context by explaining that subscriptionIds create 'authenticated customer portal deep links' and affect the returned URLs, but this is marginal beyond what the schema already provides. The baseline score of 3 is appropriate given the high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('create a customer portal session') and resource ('for a customer in Paddle'), distinguishing it from sibling tools like 'create_customer' or 'create_client_side_token' which perform different operations. It provides concrete examples of what the portal enables, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool ('Don't use this tool without checking with the user first. Avoid using before gaining explicit approval') and provides prerequisites ('Ensure those creating a customer portal session are authorized to access the customer portal'). It also implies alternatives by specifying what the portal does, helping differentiate from direct API calls for individual actions like managing subscriptions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, but the description adds valuable behavioral context beyond this. It explains default behaviors ('If the quantity object is omitted, Paddle automatically sets a minimum of 1 and a maximum of 100'), provides implementation guidance for unitPriceOverrides, and specifies the response format ('If successful, the response includes a copy of the new price entity'). This goes beyond what annotations provide without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is comprehensive but lengthy, with multiple sections that could be more efficiently organized. While all content is relevant, the extensive example and implementation advice for unitPriceOverrides (7 bullet points plus a detailed example) makes it less concise than ideal. The core purpose is clear upfront, but the structure includes substantial implementation guidance that might be better placed elsewhere.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (11 parameters, nested objects, no output schema), the description provides excellent contextual completeness. It covers required parameters, explains parameter implications, provides implementation guidance, specifies response format, and includes warnings about data requirements. This adequately compensates for the lack of output schema and provides comprehensive guidance for this creation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3. The description adds significant value by explaining the practical implications of parameter choices: it clarifies when to use 'custom' vs 'standard' types, explains taxMode options with regional examples, provides detailed guidance on unitPriceOverrides implementation strategies, and explains the default behavior when quantity is omitted. This adds meaningful context beyond the schema's technical definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'create a new price in Paddle' and explains that 'Prices describe how to charge for products.' This provides a specific verb ('create') and resource ('price'), clearly distinguishing it from sibling tools like 'get_price' or 'list_prices' which are read operations rather than creation tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context about when to use this tool: 'Always include a productId in the request to relate the price to a product.' It also offers guidance on parameter selection (e.g., 'When selecting type, choose the one that best describes the use case'). However, it doesn't explicitly state when NOT to use this tool or name specific alternatives among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds valuable behavioral context beyond annotations: it explains that the link expires after an hour, details differences between transaction types (e.g., manually-collected vs. automatically-collected), and mentions tax-reporting purposes. It does not contradict annotations, as 'retrieve' aligns with read-only behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and concise, using bullet points to organize information about transaction types and availability. Every sentence adds value (e.g., explaining link expiration, transaction distinctions, and exclusions), with no redundant or wasted text. It is front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (2 parameters, no output schema), the description is highly complete. It covers purpose, usage guidelines, behavioral details (link expiration, transaction types), and exclusions (zero-value transactions). With annotations covering safety and schema covering parameters, no significant gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear descriptions for both parameters (transactionId and disposition). The description does not add any parameter-specific semantics beyond what the schema provides (e.g., it doesn't explain transactionId format or disposition implications further). Baseline 3 is appropriate as the schema adequately documents parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'retrieve a link to an invoice PDF for a transaction from Paddle.' It specifies the resource (invoice PDF) and the action (retrieve a link), distinguishing it from sibling tools like 'get_transaction' which likely returns transaction data rather than invoice links. The description is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use this tool by detailing availability for automatically and manually-collected transactions, and explicitly stating it's not available for zero-value transactions. However, it does not explicitly compare to alternatives (e.g., when to use 'get_transaction' vs. this tool for invoice-related needs), which prevents a perfect score.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds valuable behavioral context beyond the readOnlyHint annotation. It explains that results are paginated with instructions on using the 'after' parameter, recommends a default perPage value for comprehensiveness, and clarifies the nature of the data (saved payment methods for checkout vs. subscriptions). No contradictions with annotations exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded with the core purpose, followed by usage notes and parameter guidance. Each sentence adds value, such as distinguishing from sibling tools and explaining pagination. It could be slightly more concise by integrating some details, but overall it avoids redundancy and is efficiently organized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of an output schema and the tool's complexity (6 parameters, pagination), the description provides sufficient context for effective use. It covers key behavioral aspects like pagination, filtering, and sibling tool differentiation. However, it does not detail the structure of returned payment methods, which could be helpful since there's no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the input schema fully documents all parameters. The description adds minimal semantic context, such as implying that 'addressId' and 'supportsCheckout' are optional filters, but does not provide significant additional meaning beyond what the schema already specifies. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'list payment methods for a customer in Paddle' with the specific resource 'saved payment methods at checkout.' It distinguishes from sibling tools by explicitly contrasting with 'list_transactions' for subscription-related payment methods, making the scope unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool versus alternatives: it specifies that this tool is for saved payment methods at checkout, not for subscription transactions, and directs users to 'list_transactions' for the latter. It also advises on default usage ('Use the maximum perPage by default') and when to apply filters.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds valuable behavioral context beyond the annotations (readOnlyHint=false, destructiveHint=false). It explains that only 10 destinations can be active, recommends handling storage and provisioning, warns about endpointSecretKey security, and details trafficSource options (platform, simulation, all). This enriches the agent's understanding without contradicting annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose but becomes verbose with detailed explanations of parameters and edge cases. While informative, some sentences (e.g., about Paddle recommendations and local development) could be trimmed for conciseness. It's structured but not optimally efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (7 parameters, no output schema) and rich annotations, the description is largely complete: it covers purpose, usage, parameters, behavioral constraints (active limit, security), and response details. However, it lacks explicit error handling or rate limit information, which slightly reduces completeness for a creation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3, but the description adds meaningful semantics: it explains the 'type' parameter with email/url examples and destination requirements, clarifies 'subscribedEvents' returns full event type objects, details 'trafficSource' options with real vs. simulated events, and notes 'active' boolean limitations. This goes beyond the schema's enum and description fields.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool 'will create a new notification setting (notification destination) in Paddle' with specific details about what it creates. It distinguishes from siblings by focusing on notification settings rather than addresses, adjustments, customers, etc., and mentions related tools like list_notification_setting and update_notification_setting for context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool: 'Create notification destinations to get notifications, like webhooks, for events that happen in Paddle.' It also mentions alternatives like using the dashboard or other tools (list_notification_setting, update_notification_setting) for toggling active status, and advises on local development with tunneling services.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds valuable behavioral context beyond the annotations. While annotations indicate it's not read-only and not destructive, the description explains that simulations are for testing webhooks, populate payloads with demo examples if config.entities are not provided, and the response includes a copy of the new simulation entity. It also mentions the tool's purpose in validation before real events, which provides important usage context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately front-loaded with core purpose and usage, but becomes verbose with extensive option value explanations that might be better placed in parameter-specific documentation. While all content is relevant, the length could challenge quick comprehension, though it maintains a logical flow from general to specific details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, nested objects, no output schema), the description provides substantial context. It explains the tool's role in the testing workflow, distinguishes between event types, details configuration options, and mentions the response format. The main gap is lack of explicit error handling or permission requirements, but overall it's quite comprehensive for this complex tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3, but the description adds significant semantic value. It explains the distinction between single events and scenarios, provides detailed option values for scenario simulations (e.g., paymentOutcome meanings), clarifies config object requirements, and explains what happens when config.entities are omitted. This goes well beyond the schema's technical documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a new simulation for a notification setting in Paddle, specifying it configures test webhooks for single events or scenarios. It distinguishes from sibling tools like 'create_simulation_run' by explaining this tool creates the simulation while the sibling runs it, and from other 'create_' tools by focusing on webhook testing rather than actual resource creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool: 'ideal for testing webhook implementations and validating data before sending real events' and 'create and run a simulation prior to sending real events.' It also provides clear alternatives by mentioning the webhook simulator in the dashboard and specifying that the 'create_simulation_run' tool is used after creation to send events.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already provide readOnlyHint=true, which covers the safety aspect. The description adds valuable context about what information is returned ('ID, status, and the date it was created') and clarifies that this is for retrieving existing reports rather than creating them. However, it doesn't mention potential limitations like error conditions or authentication requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is perfectly concise with three sentences that each serve a distinct purpose: stating what the tool does, clarifying what information it returns, and providing usage guidance. There is no wasted language and the structure is logical and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read operation with one parameter and readOnlyHint annotation, the description provides excellent context about purpose, usage, and return information. The main gap is the lack of output schema, but the description partially compensates by describing what information is returned. It could be more complete by mentioning error cases or response format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, with the single parameter 'reportId' fully documented in the schema. The description mentions 'by its ID' which aligns with the schema but doesn't add additional semantic context beyond what's already in the structured data. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('retrieve a report entity'), resource ('from Paddle by its ID'), and scope ('only contains information about the report, like the ID, status, and the date it was created'). It distinguishes from sibling tools by specifying this is for retrieving individual reports rather than listing or creating them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool ('Use this tool to check the status of a generated report, or to get the ID of a report') and provides a clear alternative ('to then use with the get_report_csv tool to download the CSV'). This gives the agent specific guidance on appropriate use cases and next steps.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds valuable behavioral context beyond annotations: it specifies that the link expires after 3 minutes (a critical constraint) and clarifies that reports must be ready (a prerequisite condition). However, it doesn't mention rate limits or authentication needs, leaving some behavioral aspects uncovered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently structured in three sentences with zero waste: the first states the purpose, the second specifies prerequisites, and the third adds critical behavioral detail (link expiration). Each sentence earns its place by providing essential information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (1 parameter, read-only operation, no output schema), the description is nearly complete. It covers purpose, usage guidelines, and key behavioral constraints. The only minor gap is the lack of output format details (e.g., what the link structure looks like), but this is partially mitigated by the clear purpose statement.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the single parameter 'reportId' fully documented in the schema as 'Paddle ID of the report.' The description doesn't add any additional parameter semantics beyond what the schema provides, so it meets the baseline of 3 for high schema coverage without compensating value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('retrieve a link to a CSV file') and resource ('a report from Paddle by its ID'), distinguishing it from sibling tools like 'get_report' (which checks status) and 'create_report' (which creates reports). It precisely defines what the tool does without being vague or tautological.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool ('Only returned for reports that are ready') and when not to use it (if reports aren't ready), providing a clear alternative ('The status of a report can be checked using the get_report tool'). This gives complete guidance on usage context and prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds valuable behavioral context beyond the annotations. While annotations indicate it's not read-only and not destructive, the description clarifies that it 'previews... without billing that charge,' explaining the non-destructive nature in practical terms. It also describes the response format ('includes immediateTransaction, nextTransaction, and recurringTransactionDetails'), which is helpful since there's no output schema. However, it doesn't mention rate limits, authentication needs, or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and concise. The first sentence clearly states the purpose and usage, followed by explanatory details about one-time charges and the response format. Each sentence adds value without redundancy, and the information is front-loaded for quick understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (previewing charges with multiple parameters) and the absence of an output schema, the description does a good job of explaining the tool's behavior and response. It covers the purpose, usage, and output structure. However, it could be more complete by mentioning potential errors, authentication requirements, or limitations, which would help the agent use it more effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3. The description adds minimal parameter semantics: it clarifies that 'one-time charges are non-recurring items' and that 'billingCycle is null,' which relates to the 'items' parameter. However, it doesn't provide additional context for other parameters like 'effectiveFrom' or 'onPaymentFailure' beyond what's already in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'preview creating a one-time charge for a subscription without billing that charge, typically used for previewing calculations before making changes to a subscription.' It specifies the verb ('preview creating'), resource ('one-time charge for a subscription'), and distinguishes it from sibling tools like 'create_subscription_charge' by emphasizing the preview-only nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool: 'typically used for previewing calculations before making changes to a subscription.' It also distinguishes it from alternatives by contrasting with 'create_subscription_charge' (implied by the sibling list) and clarifies that it's for 'previewing calculations before making changes,' providing clear guidance on its intended context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds significant behavioral context beyond the annotations (readOnlyHint: false, destructiveHint: false). It explains that a new notification entity is created, includes retention policies ('Notifications older than 90 days aren't retained'), error conditions, and detailed status/field explanations for debugging. While annotations indicate it's not read-only or destructive, the description provides operational details that help the agent understand the tool's effects and constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and appropriately sized. It front-loads the core purpose and usage warnings, then provides detailed behavioral information. While somewhat lengthy due to the status/origin explanations, every section adds value for understanding the tool's operation and constraints, with no redundant or wasted sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (mutation operation with specific constraints) and the absence of an output schema, the description provides comprehensive context. It explains what the tool does, when to use it, behavioral details (creation of new entity, retention limits, origin restrictions), and debugging information including status values and timestamps. This compensates well for the lack of structured output documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the single parameter 'notificationId' clearly documented as 'Paddle ID of the notification.' The description doesn't add additional parameter semantics beyond what's in the schema, but it provides context about what type of notification ID is acceptable (e.g., from delivered/failed notifications with origin 'event'). This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with specific verbs ('resend a delivered or failed notification') and resources ('notification, like a webhook notification, using its ID'). It distinguishes itself from sibling tools by focusing on notification replay rather than creation or retrieval operations, which are covered by other tools like create_notification_setting, get_notification, and list_notifications.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidelines, including when to use ('resend a delivered or failed notification'), prerequisites ('Don't use this tool without checking with the user first. Avoid using before gaining explicit approval'), and exclusions ('Only notifications with the origin of event can be replayed. Replaying a notification created for a replay isn't possible'). This gives clear context for when and how to invoke the tool responsibly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate this is not read-only and not destructive, but the description adds valuable behavioral context: it explains that Paddle creates a new simulation run event entity for the replay, describes the response structure (including status fields like pending, success, failure, aborted), and provides debugging details (payload, request.body, response.body, response.statusCode). This goes beyond what annotations provide, though it doesn't cover rate limits or authentication needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and appropriately sized. It starts with the core purpose, then provides usage context, behavioral details, and debugging information. While it's slightly longer than minimal, every sentence adds value (e.g., explaining the response structure and debugging fields). It could be slightly more concise but remains efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (replaying events with debugging), the description provides comprehensive context. It explains what the tool does, when to use it, what happens during execution (creates new entity), and details the response structure for success/failure analysis. With no output schema, the description effectively compensates by documenting the response format. This is complete for the tool's purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all three parameters (simulationId, simulationRunId, simulationEventId) with clear descriptions. The description doesn't add any additional parameter semantics beyond what's in the schema, but it doesn't need to since schema coverage is complete. Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('resend an event'), the resource ('simulation run event from Paddle'), and the mechanism ('using its ID'). It distinguishes this tool from sibling tools like 'create_simulation_run' or 'replay_notification' by focusing on replaying individual events rather than creating new runs or replaying notifications.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool: 'to retest the sending of an individual event within a simulation run rather than creating a new simulation run and sending all events again.' This provides clear guidance on the alternative (creating a new simulation run) and the specific use case (retesting individual events).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate this is a mutation tool (readOnlyHint=false, destructiveHint=false). The description adds valuable context beyond annotations: it explains what discounts do ('reduce a transaction total'), mentions successful response includes 'a copy of the new discount entity', and notes discounts can be applied to transactions/subscriptions/checkout. It doesn't cover rate limits or auth needs, but provides good behavioral insight.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized and well-structured: it starts with purpose, then usage guidelines, parameter explanations, and implementation warnings. Most sentences earn their place, though the final paragraph about successful response could be more integrated. Some redundancy exists in explaining discount applications.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex mutation tool with 13 parameters, 100% schema coverage, and no output schema, the description does well: it covers purpose, usage, key parameter semantics, and behavioral outcomes. It could better explain error cases or the full response structure, but provides sufficient context for an agent to use the tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3. The description adds significant value by explaining the semantic meaning of 'type' and 'mode' enums with examples and requirements (e.g., 'flat requires currencyCode'), and clarifies organizational context ('discounts can be added to a discount group'). This goes well beyond the schema's technical descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb ('create') and resource ('new discount in Paddle'), and distinguishes it from siblings by explaining what discounts are and their function. It explicitly differentiates from list_discount_groups for organization purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool (for creating discounts for checkout or manual application) and when to use alternatives (list_discount_groups for viewing organization). It also includes prerequisites ('Ensure you have all the information needed') and warns against fabrication.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds substantial behavioral context beyond the readOnlyHint annotation. It explains pagination mechanics ('use the 'after' parameter with the last ID'), data format details ('amounts are in the smallest currency unit'), collectionMode behaviors (automatic vs manual), and status definitions with workflow implications. This provides rich operational context that annotations alone don't cover.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is comprehensive but verbose at 22 sentences. While information is valuable, it could be more front-loaded with critical details. The lengthy explanations of collectionMode and status could be condensed, though they do earn their place by providing important behavioral context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (26 parameters, no output schema), the description provides excellent completeness. It covers pagination, filtering, sorting, data formats, include options, and detailed explanations of collectionMode and status - essentially everything needed to use the tool effectively despite the lack of output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3. The description adds meaningful context about parameter usage: it explains the 'include' parameter's purpose and what each option returns, clarifies that amounts use smallest currency units, and provides guidance on default pagination behavior. However, it doesn't fully explain all 26 parameters' interactions or edge cases.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'list transactions in Paddle' - a specific verb ('list') and resource ('transactions') with clear scope ('in Paddle'). It distinguishes from sibling tools like 'get_transaction' (singular retrieval) and 'create_transaction' (creation operation).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context with pagination guidance ('use the maximum perPage by default'), filtering capabilities, and include parameter usage. However, it doesn't explicitly state when to use this tool versus alternatives like 'get_transaction' for single transactions or other list_* tools for different resources.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, but the description adds valuable behavioral context beyond this. It explains that 'Transaction previews don't create transactions, so no id is returned' and describes the response structure ('details object that includes totals'), which clarifies the non-persistent nature and output format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded with the core purpose. Each paragraph addresses a specific aspect (purpose, alternatives, location info, parameter behaviors, output). While comprehensive, it avoids redundancy and maintains focus, though it could be slightly more condensed in the location explanation section.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (9 parameters, nested objects) and lack of output schema, the description does a good job covering key aspects: purpose, usage guidelines, parameter semantics, and behavioral traits. It explains the response format and non-persistent nature, though it could briefly mention error handling or rate limits for full completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds meaningful context for parameters: it explains the purpose of location information (tax/price calculation), lists options (customer_ip_address, address, customerId/addressId/businessId), and clarifies behaviors for includeInTotals and ignoreTrials. This enhances understanding beyond the schema's technical definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'preview a transaction without creating a transaction entity.' It specifies the verb ('preview'), resource ('transaction'), and distinguishes it from actual creation, making it distinct from sibling tools like 'create_transaction'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool versus alternatives. It states: 'Consider using the preview_prices tool for simpler pricing calculations where payment is often taken through checkout,' naming a specific sibling tool and clarifying the use case difference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds significant behavioral context beyond the annotations: it explains the asynchronous nature of report generation, status transitions (pending→ready), CSV format, expiration behavior, and the need to use get_report_csv for downloading. While annotations indicate it's not read-only and not destructive, the description provides practical operational details that aren't captured in structured fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is comprehensive but lengthy and could be more front-loaded. While all information is relevant, the detailed filter explanations (which are valuable) make it verbose. The structure moves from high-level purpose to detailed implementation specifics, which is logical but not optimally concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (asynchronous operation, multiple report types, numerous filters) and lack of output schema, the description provides complete context. It explains the workflow, available report types, filtering options, response format, and integration with other tools, making it fully self-contained for agent understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 100% schema description coverage, the description adds substantial value by explaining each report type's purpose and detailing all available filters with specific examples and constraints. It provides context about what each filter means and which report types they apply to, going well beyond the schema's basic parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a new report in Paddle, specifying it's for detailed financial data analysis, reconciliation, or export to spreadsheets. It distinguishes from sibling tools by explicitly contrasting with listTransactions for larger data gathering.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool ('when detailed financial data for analysis, reconciliation, or export to spreadsheet applications is needed') and when to prefer it over alternatives ('Use this tool over listTransactions when trying to gather larger amounts of data'). It also mentions the asynchronous nature and follow-up tools needed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds significant behavioral context beyond annotations. While annotations indicate it's not read-only and not destructive, the description details: payment processing delays ('responses may take longer than usual'), how to retrieve charge details post-creation, that charges aren't returned in the response, and specific handling of payment failures. This provides crucial operational context that annotations don't cover.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is comprehensive but verbose at 15+ sentences. While all content is relevant, it could be more front-loaded with critical information. The warning about user approval appears early, but some operational details are buried. It's appropriately sized for the tool's complexity but not optimally structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no output schema, the description provides exceptional completeness. It covers: purpose, usage warnings, parameter implications, behavioral characteristics (processing delays, response limitations), and follow-up procedures. Given the tool's financial nature and complexity, this level of detail is appropriate and helpful for safe agent operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 100% schema description coverage, the description adds valuable semantic context for parameters. It explains the practical implications of effectiveFrom choices ('next_billing_period' vs 'immediately'), clarifies onPaymentFailure behavior, and provides guidance on how to use the items parameter with catalog vs non-catalog items. This goes beyond the schema's technical definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'create a one-time charge for a subscription in Paddle' and specifies it's for 'non-recurring items' where 'billingCycle is null'. It distinguishes from siblings by focusing on subscription charges rather than general creation tools like create_transaction or create_adjustment.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use ('to bill non-recurring items to a subscription') and includes strong warnings: 'Don't use this tool without checking with the user first' and 'Avoid using before gaining explicit approval'. It also references sibling tools for follow-up actions (get_subscription, list_transactions).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![paddle-mcp-server MCP server](https://glama.ai/mcp/servers/PaddleHQ/paddle-mcp-server/badges/card.svg)](https://glama.ai/mcp/servers/PaddleHQ/paddle-mcp-server)

Score Badge

Copy to your README.md:

[![paddle-mcp-server MCP server](https://glama.ai/mcp/servers/PaddleHQ/paddle-mcp-server/badges/score.svg)](https://glama.ai/mcp/servers/PaddleHQ/paddle-mcp-server)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/PaddleHQ/paddle-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server