POD Benchmark

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations cover the safety profile (readOnly, idempotent, closed-world), and the description adds substantial behavior beyond them: unknown costs stay null, similar products stay separate, affiliate links never change rankings, totals are product+shipping, supplier-billed duty can be established/zero/bounded/unverified, profit is a pre-tax estimate from approved fee research, and estimated duty can never support a best-profit claim. That is real invocation-relevant context, not restatement.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded well — the first two sentences carry the when-to-use case — but the body sprawls into a dense run of duty/VAT/profit rules ('Show one estimated duty amount using the converted upper endpoint...') that reads like output-documentation prose rather than selection guidance. Several clauses could be compressed or delegated to the output schema without losing decision value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 12-parameter, high-complexity comparison tool with an output schema, it covers destinations, currency coupling, null handling, duty treatment and tax exclusions. The one gap is canonicalProductId — its provenance is only implied by 'Search products first' rather than stated, so an agent must infer where the ID comes from.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 83%, so the baseline is 3, but the description earns more: it explains that yourSupplier pins that supplier first and shows the per-sale difference on every other row, that sellingPrice and targetMargin are paired inputs requiring salesChannel/shopLocation for fees, that pricingTier must be explicitly requested with 'base' as standard, and that choices/optionKey fall back to the product's usual configuration whose label is returned. That cross-parameter dependency is not in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource ('compare print-on-demand supplier costs for a product') and immediately names the concrete comparison axes: product prices, shipping, delivered totals, delivery times. It also enumerates the supplier set and the product categories, so an agent can distinguish it from get_supplier_prices or search_products without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit triggers (seller asks which supplier costs less, Printify vs Printful, Etsy/Shopify fulfillment choice), explicit prerequisites ('Search products first', use get_product_options for size/color), and explicit exclusions ('Do not use for ordering, store integrations, print-quality rankings or unsupported destinations'). It also sets a re-call rule: whenever the seller mentions their supplier, selling price, margin or platform, call again with those fields.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, idempotent, non-destructive, closed-world behavior, so the safety profile is covered. The description adds a useful caveat that specs do not guarantee every supplier offers every option, but it does not discuss access requirements, rate limits, or how similarity results are limited.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences: the use case is front-loaded, followed by return scope and a caveat. Every sentence adds useful information without repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The annotations and output schema cover safety and return shape, and the description explains purpose and a caveat. However, the absence of parameter semantics for both inputs leaves an agent without guidance on the required product ID or the similarity limit.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must explain the two inputs, but it does not. It only hints at similar products and gives no meaning for canonicalProductId, the default similarLimit of 5, or the 0–20 range.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

It uses a specific verb and resource, lists the product facts returned (specs, fabric, fit, print methods), and states that similar products are included. It does not explicitly distinguish itself from siblings such as compare_product or get_product_options, so sibling differentiation is only implicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The opening sentence gives a clear trigger: use it when a seller asks about specs, fabric, fit, print methods, or similarity. It does not state when not to use it or name an alternative sibling that should be used instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, closed-world, so the safety profile is covered. The description adds real context beyond them: the prerequisite dependency on search_products and the set of supported destinations. It stops short of describing pagination/limit behavior or how many options a typical response contains.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the use case and example, then prerequisites, then the supported-destination list. Mostly earns its words, though 'Pass a returned key as optionKey' introduces a parameter name that doesn't exist on this tool and briefly muddies the otherwise tight flow.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return values needn't be explained, and annotations cover the safety profile; the description supplies the remaining pieces an agent needs — trigger, prerequisite, downstream tool, and destination coverage. Pagination guidance for the limit/offset pair is the one notable omission.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 43%, so the description has to carry more weight. It explains how to narrow results ('with words or choices') and how to feed a returned key or choices into compare_product, which adds value over the schema, but limit, offset, and the canonicalProductId pattern are left undocumented anywhere.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — lists a product's fields and supported configurations — scoped by destination and currency, and immediately distinguishes itself from compare_product and search_products. An agent can tell what it returns (supplier pricing options for a size/color) without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The opening clause names the exact trigger scenario ('when a seller wants supplier pricing for a specific print-on-demand product size or color') and gives a concrete example. It also spells out the prerequisite ('Search products first to obtain canonicalProductId') and the downstream handoff to compare_product.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, idempotent, non-destructive, closed-world, so the description isn't burdened with safety disclosure. It adds genuine context beyond them: facts carry verification date and sources, and location data is reported not guaranteed — a real caveat an agent would otherwise get wrong.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The usage trigger is front-loaded and followed by a dense list of covered topics. The second sentence is long but every clause (verification date, sources, location caveat) carries information; no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values needn't be described, and the fact categories are covered well. Missing for a 6-parameter tool is any guidance on paging (limit/offset), on the 'provider' parameter or how it interacts with 'include', leaving parts of the contract undocumented.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 33%, and the description compensates only partially: it names the categories that map to 'include' and adds meaning to location data. limit, offset, provider, and the supplierId constraint are left entirely to the schema; pagination behavior is never explained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb and resource ('how a supplier works') and it enumerates concrete fact categories: plans and monthly fees, custom branding, checkout, support, refunds, printing locations, print providers. It doesn't distinguish itself from siblings like supplier_guide, list_suppliers, or get_supplier_prices, so an agent must infer the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

A clear trigger scenario is stated ('Use this when a seller asks how a supplier works') followed by an explicit content checklist. However, no alternative is named and no exclusion is given, so the agent gets a positive condition but no routing rule against supplier_guide or get_supplier_prices.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, closed-world behavior, so the safety profile is covered. The description still earns credit for disclosing the pricing basis (each product's usual configuration delivered to the destination) and that checkedAt is the supplier's latest scrape. It says nothing about result volume, pagination caps, or what happens when a supplier lacks a product.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two dense sentences: the first front-loads the triggering situation and the questions answered, the second supplies the pricing basis and field semantics. Nothing is repeated from the schema and no sentence is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists and annotations cover safety, so the description need not explain return shapes. However, for an 11-parameter tool the omission of pricingTier semantics (base/tier1/tier2, which materially changes every price) and of currency defaulting behavior leaves real ambiguity that the schema alone does not resolve.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 36% schema coverage across 11 parameters, the description partially compensates by explaining versus (one supplier instead of all), category, and the destination basis of prices. It leaves limit, offset, search, currency, pricingTier, and categoryLimit undocumented, so an agent must infer the meaning of pricingTier and the pagination pair from bare enum names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a concrete resource (a supplier's prices) and a concrete output shape: how many products it is cheapest for, where it costs more, and by how much. It is clearly about whole-supplier rollups rather than single-product lookups, though it never names compare_product or get_supplier to make the separation explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Use this when a seller asks how a supplier's prices compare overall, by category, or against one other supplier" gives three concrete triggering scenarios that map onto the category and versus parameters. It stops short of stating when NOT to use it or pointing at the sibling tools for per-product or supplier-metadata questions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is fully covered structurally. The description adds that returned prices are destination-scoped and delivered, but says nothing about pagination limits, result ordering, or what happens when offset exceeds the result set.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no filler, and the core purpose is front-loaded ahead of the sibling handoff. Every clause carries information the agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a read-only, idempotent tool, an output schema present, and annotations covering safety, the description only needs to establish purpose and routing, which it does. The main residual gap is guidance on using the parent parameter for hierarchical traversal, which the description never mentions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only 'destination' is reflected in the description ('starting delivered prices for a destination'), and schema coverage is 50%, with currency and parent documented in the schema. The description gives no explanation of query, limit, or offset semantics, so pagination and search behavior must be inferred from defaults and bounds alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource (browse POD Benchmark's product categories) and states what the results carry: product counts and starting delivered prices for a destination. It is clearly distinct from siblings like list_suppliers or get_product, though the verb 'browse' is softer than the actual list-with-pagination behavior.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It sets the entry-point context ('use this to browse ... categories') and gives an explicit downstream handoff: pass a category id to search_products to list its products. It does not cover when NOT to use it or how to traverse the hierarchy via the parent parameter, but the primary workflow routing is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false, and closed-world scope, so the safety profile is fully covered. The description adds only the returned fields (type and website), and says nothing about pagination limits or how the 'query' filter behaves.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

One short, front-loaded sentence with no filler; every clause ('list', 'suppliers POD Benchmark compares', 'type and website') earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The presence of an output schema lowers the need to explain return values, and annotations cover the safety profile. However, with three undocumented pagination/filter parameters, the description leaves an agent guessing about paging behavior and query semantics.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for three parameters (limit, query, offset), so the description must compensate and does not: it never mentions pagination, result caps, or what 'query' searches. An agent cannot infer whether query matches supplier name, type, or website.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('list the print-on-demand suppliers POD Benchmark compares') and adds the returned fields (type, website). It implicitly distinguishes itself from the singular get_supplier, but does not name siblings or explain the difference explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Use this to list...' implies a browsing/overview use case, but gives no explicit when-to-use versus get_supplier or search_products, and no exclusions. Adequate minimum-viable guidance only.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only, idempotent, non-destructive, and closed-world. The description adds real value beyond that by stating 'It never opens the link' — a key behavioral fact that explains why the URL must be a recognizable supplier page — and by describing the return content (matching product plus any selected colour). It does not say what happens when a link is unrecognized, which keeps it from a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the when-to-use clause, then the return value, then the follow-on tool. Nothing is repeated from the schema or annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description correctly does not re-document return structure, yet still calls out the salient result (matching product + colour). The only gap is failure behavior for unsupported or unrecognized links, which an agent would likely need to set expectations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and there is only one parameter, so the baseline is already high. The description still adds meaning the schema's prose ('such as a Printify or Printful product link') does not: the full list of accepted suppliers, which is the actual semantic constraint on the url argument.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource: resolve a supplier product link into a POD Benchmark product, with the supported supplier platforms enumerated. This clearly separates it from search_products or get_product, which take identifiers rather than pasted links.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit trigger condition ('when a seller pastes a supplier product link') plus the exact follow-on action ('ready for compare_product with yourSupplier set to that supplier'). The agent knows both when to call it and what to do with the result.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this as a read-only, idempotent, non-destructive, closed-world search, so the safety profile is covered. The description adds genuine value by disclosing return content (product ids, starting delivered price, optional facet counts) and supported destinations. It says nothing about ordering defaults (relevance), pagination via limit/offset, or result-set size limits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the usage condition, and every clause carries operational content — no filler. The supplier roster is slightly long but is also the domain definition, so it earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 15-parameter, zero-required search with a nested specs object and an output schema, the description supplies the conceptual model (browse vs query, filter axes, destination-scoped pricing) and defers return-shape detail to the output schema. What's missing is pagination/ordering behaviour and how filters combine (AND vs OR).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 60% schema coverage the description meaningfully supplements: it explains that query accepts product type, brand or model with concrete examples (Bella+Canvas 3001, Gildan 5000), and enumerates the filter dimensions (category, brand, supplier, audience, specs, weight) that map to distinct parameters. It omits sort, currency, pricedOnly and pagination semantics, which remain schema-only.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('search_products') scoped to POD products across named suppliers, which is far more precise than the title alone. It implicitly differentiates from compare_product, though it doesn't clarify its boundary against get_product / get_product_options.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Opens with an explicit trigger ('Use this when a seller wants to find a POD product') and gives two concrete entry paths: free-text product type/brand/model, or browsing with filters. It names compare_product as the tool for size/colour-level comparison, but gives no guidance on when to prefer get_product instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, closed-world behavior, so the safety profile is covered. The description adds genuine value by disclosing the three-way fit triage, that reasons are dated and sourced, that a 'need a check' state exists (partial/uncertain data), and that results are not AI-ranked. It does not explain the shopLocation vs destination interaction or result ordering/cost of a wide query.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the usage trigger, followed by return-shape and a useful 'No AI ranking' caveat; there is little filler. The middle sentence is long and packs criteria into a comma chain, which slightly reduces scannability but does not waste words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With annotations covering safety and an output schema covering return values, the remaining burden is usage and parameter meaning, and the description handles most of it. The notable gap is the unexplained relationship between destination (buyer) and shopLocation (seller), which affects eligibility results; pagination is also unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 22%, so the description must compensate; it does so partially by mapping seller criteria to flags ('whether they need custom branding', 'want to start free', 'want to choose their print provider') and listing the product types. However, it only describes buyer location ('where buyers are') and leaves shopLocation's distinct meaning unclear, and it says nothing about limit/offset pagination over a potentially large result set.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the resource (suppliers) and the specific operation (matching suppliers to a seller's product type and shop constraints), and it spells out the return shape: three fit buckets plus a delivered price range. It never names a sibling tool (list_suppliers, get_supplier), so the boundary is inferred rather than stated, but the matching semantics are clear enough to separate it from the lookup tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit trigger ('Use this when a seller is choosing a supplier...') and enumerates the decision axes (buyer location, platform, branding, free start, provider choice) that make the tool appropriate. It stops short of exclusions or naming alternatives such as get_supplier_prices when the agent only needs pricing, so there is no when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.