Free open check: can an AI agent reach checkout on this store? Advisory; ARC stops before payment.
Scored across 18 tools
Several tools target nearly the same question — 'can an agent check out at this store?' — across precheck, can_agent_checkout, ask_arc, and precheck_bulk, and the descriptions themselves have to spend sentences disclaiming overlap ('for any store URL, call precheck first'). Adjacent read tools (get_agent_facts, product_data_check, google_checkout_readiness, store_card) are also similarly shaped saved-HTTP-read endpoints. The descriptions do successfully separate them (brand-in-index vs. arbitrary URL vs. bulk vs. Google-specific vs. typed facts vs. policies), so boundaries exist but require careful reading.
All names are snake_case, which is good, but the verb pattern is mixed: verb-led names (get_agent_facts, list_category_agent_facts, find_agent_ready_stores, submit_store, validate_ucp_profile, ask_arc) coexist with bare noun names (precheck, store_card, protocol_matrix, dataset_info, agent_leaderboard, product_data_check, google_checkout_readiness). precheck/precheck_bulk pair nicely, but overall there is no predictable verb_noun convention to infer a tool's shape from its name.
At 18 tools this sits in the heavy-but-explainable band; each tool maps to a distinct data product or API endpoint (dataset, leaderboard read/submit, UCP validation, protocol matrix, benchmarks). However, a meaningful chunk of the surface is near-duplicate store-check variants that could plausibly be folded into precheck with parameters, pushing it past 'well-scoped'.
The domain is covered end to end: single and bulk prechecks, arbitrary-URL and index-brand lookups, agent facts, product/policy data, Google readiness, UCP schema validation, dataset snapshot info, leaderboard read plus submission, store submission, protocol matrix, adoption stats, and benchmark pilot. Read, write/submit, and validation paths all exist with no obvious dead ends.
Read ARC's agent commerce adoption counts, category and commerce platform rates from saved HTTP checks, including Checkout WebMCP among measured Shopify checkouts, with unknown and not checked separate; shopper approval required. Curated sample, not all e-commerce. No store requests or model calls.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive, closed-world behavior, so the bar is lower. The description still adds real behavioral context: the data comes from saved HTTP checks with no live store requests or model calls, and unknown/not-checked values are kept separate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the purpose, but the single semicolon-chained sentence packs scope, methodology, an odd 'shopper approval required' clause, and provenance into one dense run-on, which hurts scannability without being truly verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description usefully characterizes what is returned (adoption counts, category and platform rates, Checkout WebMCP among Shopify checkouts, separate unknown/not-checked buckets) and sets scope limits, which is enough for a zero-parameter read tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters and schema coverage is 100%, so there is nothing for the description to disambiguate. Baseline 4 applies with no gap to compensate for.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Read) and a clear resource (ARC's agent commerce adoption counts, category and commerce platform rates) plus the data source (saved HTTP checks). It is distinguishable from siblings like google_checkout_readiness or product_data_check, though it never names them to sharpen the boundary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied rather than stated: the scope caveats ('Curated sample, not all e-commerce', 'No store requests or model calls') tell the agent what the data does and does not cover, but there is no explicit when-to-use-this-vs-alternative routing among the many siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read ARC's open agent leaderboard: cart success over the fixed ARC-100 v1 test set for each agent path ARC measures (ARC browser shoppers A-E, the UCP path, the storefront MCP path), with n, dates, evidence labels and Wilson 95% intervals, plus unverified outside submissions. Saved results only; no store requests or model calls. CC-BY-4.0.
| Name | Required | Description | Default |
|---|---|---|---|
| include_stores | No | Also return the per-store result grid (100 rows). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive and closed-world, so the safety profile is covered. The description adds genuinely non-redundant context: that results are saved-only with no store requests, that the dataset includes unverified outside submissions (a data-reliability caveat), and that output carries evidence labels and Wilson 95% intervals plus CC-BY-4.0 attribution terms.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the verb and resource, and nearly every clause carries information (scope, agent paths, returned statistics, saved-only constraint, license). The middle is a dense run-on sentence, and the trailing "CC-BY-4.0" is a bare license token, so it is efficient but not maximally clean.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description takes on the burden of describing returns and does so: n, dates, evidence labels, Wilson 95% intervals, unverified outside submissions, and per-agent-path breakdowns. Combined with annotations covering the safety profile and the schema covering the one parameter, an agent has everything needed to call and interpret this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is a single optional parameter (include_stores) and schema description coverage is 100%, so the schema already documents the per-store grid toggle. The description adds nothing about this parameter, which is the expected baseline when the schema does the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ("Read ARC's open agent leaderboard") and immediately scopes the content: cart success over the fixed ARC-100 v1 test set for the enumerated agent paths (browser shoppers A-E, UCP, storefront MCP). The "Saved results only; no store requests or model calls" clause separates it from live-check siblings like can_agent_checkout and precheck without the agent needing to open another schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a clear selection condition: this returns saved, precomputed results rather than issuing store requests or model calls, which routes the agent away from live-probe siblings. It does not name any alternative explicitly (e.g. submit_leaderboard_run for writes, or a live-check tool when fresh data is needed), so the routing is implied rather than spelled out.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read whether an AI agent can reach checkout for a brand the ARC Index already lists. Use it for 'is agent-ready?', 'how does score on ARC?' or 'which brands do agents get furthest with?'. For any store URL, call precheck first. No login and no new scan. Pass a brand, URL, or category. Returns the score, evidence, and both lanes: the agent-catalog HTTP check and the browser cart. The JSON field agents_can_buy is true when the catalog returned a checkout link or a browser shopper added the product to the cart. ARC stops before payment. It is not a completed order. The catalog check does not change the score. A missing score is not a zero. Results are advisory.
| Name | Required | Description | Default |
|---|---|---|---|
| q | No | Free-text brand, product, or category query. | |
| url | No | Brand, store, or product URL already on the ARC Index. | |
| brand | No | Brand name, e.g. Allbirds. | |
| category | No | Index category or shopper words, e.g. footwear, running shoes, leggings. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so well: no login, no new scan, two distinct lanes (catalog HTTP check and browser cart), the exact semantics of `agents_can_buy`, the boundary that ARC stops before payment and is not a completed order, that the catalog check does not alter the score, and that a missing score is not a zero. These are exactly the non-obvious behaviors an agent needs before interpreting results.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose and the routing rule are front-loaded, and the trailing caveats ('not a completed order', 'missing score is not a zero', 'results are advisory') each carry real interpretive value. The text is dense but a few caveats could be compressed without losing meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description must explain returns, and it does: score, evidence, and both lanes. Combined with the interpretation caveats and the precondition on store URLs, an agent has everything needed to call it and read the answer correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds meaning beyond the schema by framing brand/URL/category as alternative entry points ('Pass a brand, URL, or category') and constraining URL to one already on the ARC Index. The `q` free-text parameter is never mentioned in the description, keeping it from a 5.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: read whether an AI agent can reach checkout for a brand already on the ARC Index, including the scope restriction that the brand must be indexed. It distinguishes itself from `precheck` explicitly ("For any store URL, call precheck first") but leaves its relationship to `can_agent_checkout`, `find_agent_ready_stores`, and `agent_leaderboard` unstated, which are the most likely confusable siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides concrete user-utterance patterns ('is <brand> agent-ready?', 'how does <brand> score on ARC?', 'which <category> brands...') plus an explicit routing rule to a named alternative for the store-URL case. No exclusion guidance against the other readiness-related siblings, so it stops short of full when/when-not coverage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Call when you need a saved checkout-link check for any public store URL, not only Index brands. Use it for 'what stops an agent at checkout?' or 'which checkout route works on ?' when you also want the one fix. Saved HTTP checkout check for a public homepage or product URL: catalog, product, checkout link, UCP profile, blocks, and one fix. No login, model call, payment, or browser scan. Uses ARC's last check (cached up to 24 hours). A missing result is unknown, not a no. A checkout link means reach checkout, not a completed order. Results are advisory. ARC stops before payment.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Public store homepage or product URL |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations it discloses non-obvious behavior: the result is a cached ARC check up to 24 hours old, no login/model call/payment/browser scan occurs, a missing result means unknown rather than 'no', a checkout link means reachable checkout not a completed order, and output is advisory with ARC stopping before payment. These are exactly the interpretive caveats an agent needs and none are recoverable from readOnlyHint/openWorldHint/idempotentHint.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The invocation condition is front-loaded in the first sentence and every subsequent sentence adds a distinct fact (what the payload contains, what it does not do, cache freshness, interpretation caveats). It is dense and the first sentence is a comma-heavy run-on, but there is no filler to cut.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With one required param, full annotations, and no output schema, the definition still describes the returned fields (catalog, product, checkout link, UCP profile, blocks, fix) and explains how to read a null result. Nothing an agent needs in order to call it correctly or interpret the response is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the single parameter's description already says 'Public store homepage or product URL', so the schema carries most of the load. The description adds real value by clarifying the URL may be any public store, not only Index brands, which prevents an agent from withholding a valid URL; it does not explain malformed-URL handling or homepage-vs-product differences.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The definition names a specific operation (saved HTTP checkout-link check) and resource (a public store homepage or product URL), and enumerates what the result contains: catalog, product, checkout link, UCP profile, blocks, and one fix. It even scopes itself against a narrower variant ('not only Index brands'), though it never names siblings like google_checkout_readiness or precheck, so sibling disambiguation is only partial.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit trigger conditions in natural-language form ('Call when you need...', 'what stops an agent at <store> checkout?', 'which checkout route works on <store>?'), which is strong positive guidance. It lacks any explicit when-not or named-alternative routing (e.g. 'use google_checkout_readiness instead for X'), so an agent must infer the boundary from the sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Latest open ARC dataset snapshot (start with store_answers: one plain answer per store on whether AI agents can find products and get a checkout link): immutable versioned download URLs plus latest/ aliases (CSV and Parquet), row counts, method version and licence (CC-BY-4.0, attribution ARC, arcreport.ai). Reads only.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and closed-world, so 'Reads only' adds nothing. However, with no output schema the description usefully discloses what comes back: immutable versioned URLs plus latest/ aliases in two formats, row counts, method version, and licence terms (CC-BY-4.0, attribution ARC). That return-shape detail is real value beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single dense sentence, front-loaded with the resource and its contents. The store_answers parenthetical is the one slightly off-topic insertion, but it is short and does not bury the core payload.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless metadata tool with no output schema, the description covers the essentials an agent needs: what the snapshot is, what fields it exposes, the formats and alias scheme, and the licence. It does not mention pagination or size limits, but those are largely inapplicable here.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is nothing for the description to disambiguate; the baseline for a parameterless tool applies. No parameter claims are made that could mislead.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific resource ('Latest open ARC dataset snapshot') and enumerates its contents (immutable versioned download URLs, latest/ aliases in CSV and Parquet, row counts, method version, licence). It is clearly distinguishable from the sibling tools, which all concern stores, agents, or leaderboards rather than dataset metadata, though it never states a verb like 'returns' or 'describes'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The parenthetical 'start with store_answers: one plain answer per store on whether AI agents can find products and get a checkout link' gives a starting point for analyzing the dataset, but it does not say when an agent should call dataset_info versus siblings such as ask_arc or get_agent_facts. Usage is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Find stores where a saved check shows an AI agent can reach checkout, for running shoes, beauty, clothing, home goods, or another product need. Use it for 'where can my agent check out for running shoes?' or 'which stores let an agent reach checkout for ?'. Reads real HTTP agent-catalog checks from the last thirty days whose catalog found a product and returned a checkout link. This is not a completed order. One tested product is not a promise about another item. No login, browser scan, or model call. Results are advisory. ARC stops before payment.
| Name | Required | Description | Default |
|---|---|---|---|
| q | No | Product need, such as skincare or camping gear | |
| limit | No | ||
| category | No | Category slug or shopper words such as running shoes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the readOnlyHint/idempotentHint annotations: data source is real HTTP agent-catalog checks from the last thirty days, an inclusion requires a found product plus a returned checkout link, no login/browser scan/model call is performed, results are advisory, and the flow stops before payment. The caveats that it is not a completed order and one tested product does not generalize are exactly the kind of context annotations cannot carry.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action and the query examples in the first two sentences, which is the right ordering. It is slightly over-stacked with near-repetitive disclaimers ('Results are advisory', 'This is not a completed order', 'ARC stops before payment') that could be condensed into one clause.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a three-parameter read-only lookup with rich annotations, the description covers eligibility criteria, data recency, and safety boundaries thoroughly. The remaining gap is that no output schema exists and the description never indicates what a result row contains (store name, checkout link, check date), which an agent will want before presenting results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 67% and the schema already documents q and category, so baseline is 3. The description reinforces category scope with examples but never clarifies how q and category interact (union, precedence, or fallback) or what limit controls, leaving the only real semantic ambiguity unresolved.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource with its qualifying condition: finds stores whose saved check shows an AI agent reached checkout. It is clearly differentiated from point-check siblings like can_agent_checkout or precheck by being a discovery/list operation across categories.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives two concrete sample phrasings and a scope condition ('for running shoes, beauty, clothing, home goods, or another product need'), so an agent knows the intended moment. It does not explicitly name sibling alternatives (can_agent_checkout, precheck, store_card) or when to prefer them, keeping it below a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read free ARC agent facts for a store or product with no login. Returns TypeSafe Jev typed decisions (use-case mention, price clarity, agent-friendly description, cart-blocker suspicion, readiness score) from an HTTP scrape. Not a browser session and not any third-party agent product; ARC's browser cart tests (ARC browser shoppers A–E) are separate. JSON: GET /api/agent-facts?host=
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | Store or product URL | |
| host | No | Store hostname, e.g. allbirds.com | |
| slug | No | ARC Index slug, e.g. allbirds |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does reasonably well: it discloses access requirements (no login, free), the mechanism (HTTP scrape rather than a browser session), and the semantic payload returned. It does not mention rate limits, caching, failure modes, or cost ceilings, so it is not exhaustive, but the auth/mechanism profile is genuinely useful.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the verb+resource, then the return payload, then exclusions, then the endpoint. Three sentences, all earning their place, though 'TypeSafe Jev typed decisions' is jargon that adds little for an agent and could be trimmed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, and the description compensates by enumerating the returned fields (use-case mention, price clarity, agent-friendly description, cart-blocker suspicion, readiness score). Combined with the auth and mechanism disclosure, an agent has enough to invoke correctly; only ambiguity about which of the three parameters to supply keeps it from a 5.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so url, host, and slug are already documented in the schema; baseline is 3. The description only adds a sample endpoint fragment ('GET /api/agent-facts?host=') that hints at the host parameter, with no guidance on precedence between url/host/slug.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Read) and resource (ARC agent facts) scoped to a store or product, and explicitly names what it is NOT: a browser session, a third-party agent product, or ARC's browser cart tests (shoppers A–E). An agent can distinguish it from checkout-oriented siblings like can_agent_checkout or precheck without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear usage context ('free', 'no login', HTTP scrape) and an explicit exclusion ('Not a browser session... browser cart tests are separate'), which steers agents away from the wrong tool class. It stops short of naming the specific sibling to call for browser cart tests, so the routing guidance is implied rather than complete.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
ARC Agent Shopping Index pilot results (pilot, not a ranking): whether 3 models, run as benchmark personas via OpenRouter (not the consumer apps), reached a checkout step on their own on 35 non-Shopify stores, stopped before payment, with no accounts. Per model: hand-verified checkouts out of n with Wilson 95% intervals, cart or better, blocks, median steps and time, cost per checkout; plus every store × model outcome. One run, one reviewer, and the intervals overlap. Returns {"status":"draft"} until the pilot is published. Reads only.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the read-only and idempotent annotations, the description discloses key runtime behavior: it returns {"status":"draft"} until publication, the data is a single run reviewed by one person, and confidence intervals overlap. These are material caveats an agent would not learn from the annotations alone.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The first clause front-loads what the tool returns and the pilot/ranking distinction, and the rest lists the returned metrics efficiently. It is dense and long for a single sentence but every clause conveys relevant scope or return-content detail; minor trimming of statistical field names could improve it.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by enumerating the returned content: hand-verified checkouts with Wilson intervals, cart-or-better outcomes, blocks, median steps and time, cost per checkout, and store × model outcomes. It also states the draft status behavior, leaving little an agent needs to know before calling it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so the baseline is 4; no parameter documentation is needed. The description adds no parameter information, but none is applicable.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific resource (ARC Agent Shopping Index pilot results) and immediately scopes it with 'pilot, not a ranking,' which distinguishes it from the leaderboard sibling. An agent can tell this is a read of pilot benchmark data rather than a ranking or a store-level check.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It clearly frames the context in which the data applies (3 models, OpenRouter personas, 35 non-Shopify stores, one run, one reviewer) and warns that intervals overlap and it is not a ranking, which implicitly routes ranking questions elsewhere. It does not explicitly name alternatives such as agent_leaderboard, so it falls short of a full when/when-not statement.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read saved public Google agentic checkout readiness checks for a store. Free; no new scan or paid calls. Merchant Center feed eligibility cannot be checked from outside. Not Google approval or certification.
| Name | Required | Description | Default |
|---|---|---|---|
| host | Yes | Store host or URL |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden and does well: it discloses cost (free), that no new scan or paid calls are made, what cannot be verified (Merchant Center feed eligibility), and that results are not Google approval or certification. It omits how fresh or how scoped the returned checks are, keeping it out of 5 territory.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four short sentences, front-loaded with the core purpose and followed by cost and limitation caveats. Every sentence carries a distinct constraint, though the caveat list is dense enough to be slightly more informational than the core action warrants.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter, annotation-free, no-output-schema retrieval tool, the description covers purpose, cost model, and key limitations well. Its main gap is not characterizing the nature or freshness of the returned 'saved checks', which an agent might want when deciding between this and a live scan.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single 'host' parameter, so the schema already documents it as 'Store host or URL'. The description adds no format, normalization, or example detail beyond that, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Read saved public Google agentic checkout readiness checks for a store.' The word 'saved' crucially marks it as retrieval of existing results rather than a trigger for a new scan, which distinguishes it from scanning siblings like precheck or can_agent_checkout. It stops short of naming a sibling explicitly, so a 4 rather than 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives clear usage context: free, no new scan, no paid calls, implying the tool is for inspecting existing readiness data rather than initiating analysis. Exclusions are present (Merchant Center feed eligibility can't be checked, not an approval/certification), though it doesn't route the agent to a specific alternative tool for those cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List free ARC agent facts by category (beauty, footwear, apparel, …) with no login. Ranked by newest real Jev scores. JSON: GET /api/agent-facts?category=beauty
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | 1–50, default 20 | |
| category | No | Category id or words, e.g. beauty, running shoes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral burden and does so reasonably: it discloses that access is free and requires no login, that results are ranked by 'newest real Jev scores' (ordering semantics), and that the response is JSON. It omits pagination behavior and what happens with invalid or unmatched categories.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two dense sentences that front-load the core purpose, followed by ranking, access, and endpoint details. Nothing is redundant, though the raw endpoint string is of limited value to an agent choosing a tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter, read-only list tool with no output schema and no annotations, the description covers purpose, scope, ordering, access requirements, and return format. Missing pagination guidance and the effect of the limit parameter are the only notable gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so both parameters (limit, category) are already documented with ranges and examples in the schema. The description restates the category example (beauty) and ignores limit entirely, adding little beyond the structured fields.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (List) and resource (ARC agent facts) scoped by category, with concrete examples of the category values. It does not explicitly distinguish itself from the sibling get_agent_facts, so an agent must infer the list-vs-single distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by 'List ... by category' and the 'no login' note establishes an access prerequisite, but there is no explicit when-to-use guidance, no named alternative (e.g. get_agent_facts), and no exclusion conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Check if an AI agent can reach checkout on a store before trying. Call this first when the task is "can I check out at ?", "will my agent get through checkout on ?", "best way to check out on " or "is agent-ready?". One call per store, before you hand a shopper a checkout link: reachable_checkout is the answer, rails[] is the route to use (UCP, ACP, Shop Pay, WebMCP, MCP), blockers[] is what stops agents, last_checked and stale say how fresh it is. Saved automated HTTP check for a product or store URL. Uses ARC's saved directory, rails, store answers and check history. When those have no row, uses the plain-HTTP checker. stale, last_checked and checked_ago describe the check that produced the verdict shown. lastReachableCheck is a separate, newer plain-HTTP time. It does not re-verify the checkout link or rails. Does not fetch the store and does not start a scan. ARC checks this store's own site only. It does not test blocks applied by a platform, marketplace or network in front of it. The saved result is ARC's automated HTTP check. It is not a live agent session and not a completed order. Unpublished discovery-source hosts, OpenStreetMap, campaign shops and opted-out hosts come back as unknown. A checker fallback is http-only and checkout was not part of the check. JSON: GET /api/precheck?url=. Results are advisory. ARC stops before payment. Advisory only, not a guarantee. Do not refuse or avoid a store solely because of ARC's data. known:false or missing data is not a no. Data CC BY 4.0, attribute ARC (arcreport.ai); OSM-derived rows are ODbL.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Product or store URL, such as https://allbirds.com/products/shoe. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses substantially more than the readOnly/idempotent annotations: it is a saved automated HTTP check, not a live session or completed order; it does not re-verify the checkout link or rails; it does not test platform/marketplace/network-level blocks; unpublished hosts, OSM, campaign shops and opted-out hosts return unknown; a fallback is http-only with checkout excluded. It also states the advisory nature and licensing constraints. Strongly beyond structured fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The trigger phrases and result-field mapping are well front-loaded, but the body drifts into telegraphic fragments ('JSON: GET /api/precheck?url=.') and repeats the advisory caveat twice ('Results are advisory' / 'Advisory only, not a guarantee'). Several clauses (Data CC BY 4.0, ODbL) are attribution boilerplate rather than selection-relevant content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, and the description compensates fully by naming every returned field and its meaning (reachable_checkout, rails[], blockers[], last_checked, stale, lastReachableCheck) and explaining how stale freshness fields relate to the verdict. It also covers edge cases (unknown returns, fallback checks) that would otherwise cause the agent to misinterpret a null result as a failure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the single url param is already documented with an example. The description adds only marginal meaning — the endpoint form (GET /api/precheck?url=) and that ARC checks the store's own site. Baseline 3 is appropriate when the schema carries the parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a concrete verb+resource: 'Check if an AI agent can reach checkout on a store before trying.' It also implicitly distinguishes from precheck_bulk ('One call per store'). It never names a sibling tool explicitly (can_agent_checkout, precheck_bulk), so an agent must infer the boundary rather than being routed to it.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit trigger phrasings ('can I check out at <store>?', 'is <store> agent-ready?') and a sequencing rule ('Call this first', 'before you hand a shopper a checkout link'). It also caps usage ('One call per store', 'Does not fetch the store and does not start a scan') and warns against misusing negative results. This is exactly the when-to-use guidance an agent needs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
The precheck read for up to 100 store domains or URLs in one call. Returns one precheck object per input, in order, each with an input field, plus routes[]: the best known ways in for an agent (ucp, ucp_mcp, storefront_mcp, cart_link, browser), best evidence first, each with checked_at, age_hours and stale. Saved checks only. Does not fetch any store and does not start a scan. Each domain counts as one read against the same limit as precheck (60 a minute per network); domains past the limit come back with reason rate_limited and retry_after_seconds. More than 100 domains is refused. not_recorded and known:false are not a no. JSON: POST /api/precheck/bulk with {"domains": [...]}. Advisory only. ARC stops before payment. Data CC BY 4.0, attribute ARC (arcreport.ai).
| Name | Required | Description | Default |
|---|---|---|---|
| domains | Yes | 1 to 100 store domains or URLs, such as allbirds.com or https://allbirds.com/products/shoe. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations cover the safety profile (readOnly, idempotent, non-destructive, closed-world), and the description adds substantial non-redundant behavior: results come from saved checks only, no fetch or scan, per-domain counting against a shared 60/min limit, partial results returning reason rate_limited with retry_after_seconds, and >100 inputs being refused. It even forecloses a likely misread ('not_recorded and known:false are not a no') and states the advisory-only / no-payment boundary.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Dense but front-loaded — purpose, return shape, then behavior and limits in descending priority. Some tail material (the raw POST path, CC BY 4.0 attribution, 'ARC stops before payment') is marginally useful and could be trimmed, but nearly every sentence carries operational information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the return-value burden and does so: per-input objects with an input field and a routes[] list enumerating the route types (ucp, ucp_mcp, storefront_mcp, cart_link, browser) and their evidence/staleness fields. Combined with the rate-limit and failure semantics, an agent has what it needs to call this correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter is fully described in the schema, so the baseline is 3, but the description adds real meaning beyond it: each domain consumes one read of the shared rate-limit budget, ordering of results matches input order, and more than 100 is rejected rather than truncated.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb and resource plus scope: 'the precheck read for up to 100 store domains or URLs in one call', and immediately states the return shape (one object per input, in order). It implicitly and explicitly separates itself from the single-domain sibling by naming 'precheck' as the shared rate-limit counterpart and framing the bulk case as many-per-call.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear when-not guidance: 'Saved checks only. Does not fetch any store and does not start a scan', so an agent knows this is not a live scan trigger. Rate-limit behavior and the >100 refusal are spelled out, but it never explicitly says 'use precheck instead for a single domain', leaving the sibling routing to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Free latest saved product data and return/shipping policy check from the store's pages and structured data. Reads history only; no store fetch or model call. A missing result is not a zero.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | ||
| host | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the safe read-only, idempotent, non-destructive, closed-world profile. The description goes beyond them by disclosing that it consults stored history only (no live fetch, no model call) and by warning that a missing result is not a zero — genuinely useful interpretation guidance the agent would otherwise guess wrong about.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, each carrying distinct information (what it reads, what it does not do, how to read an empty result), with the core purpose front-loaded. Slightly awkward grammar in the opening clause but no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and only mild annotation coverage, the description adequately covers behavior and result interpretation, and the read-only annotations cover safety. The gap is parameters: url vs host is unexplained for a tool whose only inputs are those two, leaving the agent unsure how to target the check.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Two parameters (url, host) with 0% schema description coverage, and the description never mentions either one, their relationship, or whether both are needed. With this coverage level the description should compensate and it does not.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names the resource (saved product data plus return/shipping policy) and the source (store's pages and structured data), so an agent can grasp roughly what it returns. However, the framing is a noun phrase rather than a clear verb, and it never distinguishes itself from the many sibling store/agent analysis tools. Purpose is understandable but muddled.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It states constraints ('reads history only; no store fetch or model call') which hint at when it is appropriate, but there is no explicit when-to-use vs alternative and no siblings are referenced. An agent gets a boundary condition, not real routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read ARC's neutral Protocol Matrix: for UCP, ACP, PAP, WebMCP, MCP and agent payment protocols, how many checked stores declare, validate, answer and transact, with evidence labels (Measured / Inferred / Not measured). Saved checks only; no store requests or model calls. CC-BY-4.0.
| Name | Required | Description | Default |
|---|---|---|---|
| protocol | No | Optional: one protocol id. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover safety (readOnlyHint, idempotentHint, destructiveHint=false, openWorldHint=false), but the description adds genuinely new behavioral facts: it reads only saved checks, issues no store requests or model calls, and labels each figure as Measured / Inferred / Not measured. The license note (CC-BY-4.0) is a minor extra.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single dense, front-loaded sentence that covers scope, dimensions, evidence labels, data source and license in ~45 words. No filler, though the compressed phrasing (four verbs in one clause) takes a moment to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the return-value burden and does so reasonably: it says the output is counts per protocol plus evidence labels. The only omission is output shape/pagination detail, which is minor for a single-parameter read-only aggregate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a single optional enum parameter, so the schema already documents the protocol id fully. The description mirrors the enum values (UCP, ACP, PAP, WebMCP, MCP, payments) but adds no syntax or behavior detail beyond what the schema states. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Read) and a specific resource (ARC's neutral Protocol Matrix), then enumerates exactly which protocols it covers and what dimensions it reports (declare, validate, answer, transact). It is clear what the tool returns, though it never names or contrasts itself with siblings like agent_commerce_adoption or precheck.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The clause 'Saved checks only; no store requests or model calls' tells the agent the data source is pre-computed, implying it should be used for aggregate overview rather than live lookups. However there is no explicit when-to-use statement and no named alternative for the case where an agent wants per-store or live data.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read a public store card and saved brand claim. No outbound fetch or model calls. Brand statements are from the brand and not measured by ARC; claims never change scores.
| Name | Required | Description | Default |
|---|---|---|---|
| host | Yes | Store host or URL. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the safety profile is covered. The description adds real value beyond that: 'No outbound fetch or model calls' clarifies side effects, and 'Brand statements are from the brand and not measured by ARC; claims never change scores' discloses an important trust/scoring semantic an agent would not otherwise know.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, front-loaded with the primary action, followed by behavioral caveats. Nothing is padded, though the brand/claim sentence is dense and could be tighter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple single-parameter read tool with no output schema, the description conveys the content retrieved (store card plus saved brand claim) and the key trust caveat. It stops short of describing the return structure, but the scope is small enough that this is a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With a single parameter at 100% schema description coverage, the schema already documents 'host'. The description adds no format or syntax detail for the host parameter, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb ('Read') and resources ('public store card', 'saved brand claim'), which is enough to distinguish it from siblings like find_agent_ready_stores or submit_store. The term 'store card' is slightly opaque but the read scope is clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to use this tool versus the many sibling tools (find_agent_ready_stores, product_data_check, etc.), nor any prerequisites. The description only says what it reads, not when an agent should reach for it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Submit your agent's run against ARC-100 v1 to ARC's open leaderboard. Free. The run is listed as 'unverified' until ARC re-runs a sample. Format: arc-leaderboard-submission/v1 (see /leaderboard/spec). Runs must stop before payment. Limit 5 per network per day. Same as POST /api/leaderboard/submit.
| Name | Required | Description | Default |
|---|---|---|---|
| run | Yes | ||
| spec | Yes | ||
| agent | Yes | ||
| contact | No | Optional and private: how ARC can reach you to arrange verification. | |
| results | Yes | ||
| test_set | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond annotations by disclosing the verification lifecycle ('listed as unverified until ARC re-runs a sample'), a concrete rate limit ('5 per network per day'), a hard precondition ('runs must stop before payment'), and cost ('Free'). These are exactly the behavioral traits an agent cannot infer from readOnlyHint/destructiveHint/idempotentHint.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Six short sentences, all front-loaded with the core action first, then constraints. Every sentence carries information (cost, verification, format, precondition, limit, API mapping); a minor nit is that the format/API-mapping details could be consolidated, but nothing is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a write tool with no output schema, six parameters and nested objects, the description covers the submission preconditions, rate limit, and verification outcome that an agent needs to call it correctly. It omits auth requirements and failure behavior, but the essential calling constraints are present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 17%, so the description must carry most of the parameter burden, but it only clarifies the spec constant and the stopped_before_payment precondition. It says nothing about the results array shape, the 10–100 item bounds, the agent.path enum, or optional contact handling, leaving the low-coverage schema uncompensated.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource — 'Submit your agent's run against ARC-100 v1 to ARC's open leaderboard' — which unambiguously separates it from read-oriented siblings like agent_leaderboard. The API mapping ('Same as POST /api/leaderboard/submit') further pins down the exact operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear context for use: run against ARC-100 v1, must stop before payment, format arc-leaderboard-submission/v1, and a daily cap. It stops short of explicitly naming alternatives (e.g. agent_leaderboard for reading results) or stating when NOT to submit, so it is clear but not fully routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Add stores to ARC's free, open agent-commerce directory (CC-BY dataset). Pass url for one store or hosts for up to 50. Each new host is validated (live HTTPS, not parked, not redirecting to another brand); accepted hosts are queued for ARC's plain-HTTP rails checks (UCP, WebMCP, MCP, policies, product data) and appear in the dataset with source=submitted. No model calls, no login. Rate limited per caller.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | ||
| hosts | No | ||
| category | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations cover safety (destructiveHint=false, idempotentHint=true, openWorldHint=true), but the description adds substantial value: the validation pipeline (live HTTPS, not parked, not redirecting), the async queue for UCP/WebMCP/MCP checks, dataset visibility with source=submitted, and rate limiting. Doesn't say what a caller observes on rejection or how to track queued items.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Dense but every clause adds operational information (validation rules, queued checks, rate limit) with no filler. Slightly packed for a single paragraph.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 3-param submission tool with no output schema, the description covers validation behavior, the async follow-up process, dataset provenance, and cost/auth constraints. Missing only error/rejection semantics and any mention of the category parameter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It clarifies url vs hosts cardinality (max 50) which the schema does show, but says nothing about the 'category' enum parameter or its allowed values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Add stores to ARC's free, open agent-commerce directory') and distinguishes itself from read-only siblings like find_agent_ready_stores by being the submission entry point.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explains the url vs hosts alternative clearly ('Pass url for one store or hosts for up to 50') and clarifies no login/model calls are needed, but doesn't state when NOT to use this versus e.g. precheck or validate_ucp_profile.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Validate a Universal Commerce Protocol (UCP) business profile against the published UCP JSON Schema (current release) and spec rules: authority binding, https endpoints, orphaned extensions and, for a URL, hosting headers (no redirects, Cache-Control public max-age>=60). Pass url (store host, homepage or profile URL; /.well-known/ucp is added for a homepage) or profile (the JSON). Returns errors, warnings and info, each with a path and the spec section. Free, no login, no model call. Validation is not certification. JSON: GET /api/ucp/validate?url=
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | Store host, homepage or profile URL, e.g. allbirds.com | |
| profile | No | The profile JSON (object or string) to validate without fetching. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only cover readOnly/openWorld; the description adds real behavioral context: free, no login, no model call, no redirects tolerated, Cache-Control max-age>=60, and the important caveat that validation is not certification. It also discloses the response shape (errors, warnings, info each with a path and spec section), which no output schema provides.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads purpose, then inputs, then returns, then cost/trust caveats, then the raw endpoint. Dense and information-rich with no filler, though the long parenthetical check list and trailing 'JSON: GET /api/ucp/validate?url=' make it slightly cramped rather than crisp.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description still enumerates return contents and severities, covers auth/cost constraints, and states the external-side-effect-free nature of a read-only validation. An agent has everything needed to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds meaning beyond it: the /.well-known/ucp path append for a homepage URL and that the profile parameter validates 'without fetching'. Both parameters are covered with no ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (validate) plus a specific resource (UCP business profile) and scopes it to the published UCP JSON Schema and spec rules. Names concrete checks (authority binding, https endpoints, orphaned extensions, hosting headers), so it is clearly distinguishable from siblings like precheck or find_agent_ready_stores.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explains the two invocation modes and their selection condition ('Pass url ... or profile ...'), including the /.well-known/ucp auto-append behavior for homepages. It gives clear context but names no explicit alternative tool or when-not-to-use case.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
get_benchmarkprecheck_bulkprechecksubmit_store1 field changedInput schema / properties / category / enumPrevious value: -[
- "apparel",
- "footwear",
- "beauty",
- "home",
- "food-drink",
- "supplements",
- "outdoor",
- "kids",
- "pets",
- "electronics",
- "jewelry"
-]New value: +[
+ "apparel",
+ "footwear",
+ "beauty",
+ "home",
+ "food-drink",
+ "supplements",
+ "outdoor",
+ "kids",
+ "pets",
+ "electronics",
+ "jewelry",
+ "unknown"
+]agent_commerce_adoptionagent_leaderboardask_arccan_agent_checkoutdataset_infofind_agent_ready_storesget_agent_factsgoogle_checkout_readinesslist_category_agent_factsproduct_data_checkprotocol_matrixstore_cardsubmit_leaderboard_runsubmit_storevalidate_ucp_profileCan an AI agent buy on this French store? 500 stores tested: score, guest checkout, UCP. Free.
Free, no sign-in MCP: can AI agents reach checkout at a store? Try: Use ARC to check allbirds.com
AgentReady.market audit: can an AI shopping agent find, understand and BUY on this store? /100.
Is a website ready for AI shopping agents? Readiness score (0-100) + agent shopping simulation.
Add one secure layer between your agents and this server.
