Whether an AI agent may fetch a URL, on what terms, and at what price. Free, no key, no signup.
Scored across 3 tools
Each tool has a clearly distinct purpose: checking a URL's access terms, explaining the verdict taxonomy, and listing AI crawler tokens. There is no overlap or ambiguity between the three tools.
All tool names follow a consistent access_ + verb + noun snake_case pattern: access_check_url, access_explain_verdicts, access_list_agents. The convention is predictable and readable.
Three tools are well-scoped for a focused lookup service; each tool performs a necessary and non-redundant function. The count is at the low end but appropriate for the domain.
The core read-only workflow (check a URL, explain verdicts, list agents) is covered without dead ends. Minor gaps exist, such as no batch-checking or browsing of indexed origins, but agents can work around these with repeated calls.
Says whether an AI agent may fetch a URL, on what terms, and at what price. Reads robots.txt for a generic agent and for every AI user-agent the operators document, the Content-signal and Content-Usage directives, any RSL licence the origin links, and the live response to an honestly-identified request.
Verdicts: allowed (it served us), allowed_if_identified (the written policy permits
this but bot management refused an anonymous request — signing as a named crawler is the
documented route), charged (there is a price, quoted where the rail will show one),
blocked (the site disallows the agents that would want this, with nothing to buy), and
unknown (the origin did not answer, or the URL is not there).
Never evades: no CAPTCHA solving, no proxies, no spoofed user-agents. Refusals come back as findings with the evidence attached.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Absolute http(s) URL of the page to check, e.g. https://example.com/article | |
| skipWellKnown | No | When true, the five well-known probes are skipped; faster, and those files are almost always absent | |
| includeEvidence | No | When true, the answer includes every request made, with status and headers (default false: it is large) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safe-read profile (readOnly, idempotent, non-destructive, openWorld), yet the description adds substantial context: the specific evidence sources consulted (robots.txt, Content-signal/Content-Usage, RSL licences, live request), the 'Never evades' constraint (no CAPTCHA solving, proxies, or spoofed user-agents), and that refusals return as structured findings with evidence.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core purpose, then the evidence sources and verdict list. The verdict enumeration is essential since there is no output schema. It is longer than strictly necessary, but nearly every sentence carries information; minor trimming of the verdict glosses would help.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema in play, the description properly compensates by enumerating the five possible verdicts and their meanings, plus the guarantee that refusals surface as findings with evidence. An agent has everything needed to call the tool and interpret the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents url, skipWellKnown, and includeEvidence. The description adds marginal value by explaining that well-known probes are part of the normal work (justifying skipWellKnown), but it adds no syntax or format detail beyond the schema. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Says whether an AI agent may fetch a URL, on what terms, and at what price.' The resource (URL accessibility policy) is distinct from the siblings access_explain_verdicts (explains results) and access_list_agents (enumerates agents), so an agent can route correctly without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Clearly implies the context: check a URL before an agent fetches it, and it explains what the tool inspects plus the full verdict taxonomy so the caller knows how to interpret results. However, it never names when to prefer a sibling (e.g. access_explain_verdicts for interpreting an existing verdict), so usage is strong but not fully routed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Returns the five verdicts access_check_url can give, what each one means for an agent that has just been refused, and the next move for each, plus what each confidence level rests on. Free, makes no network request, and returns the same answer every time: it is the key to reading access_check_url results.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover safety (readOnlyHint, destructiveHint=false) and determinism (idempotentHint), but the description adds genuinely useful context beyond them: it is free, makes no network request, and returns identical output every time. That reassures an agent this is a zero-cost reference lookup rather than a live check.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single dense sentence, front-loaded with what it returns and closing with the relationship to access_check_url. Every clause carries information, though the sentence is long enough that a second short line about its static nature would read cleaner.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no parameters and no output schema, the description still enumerates the return contents (five verdicts, meanings, next moves, confidence bases) and the cost/determinism profile. Nothing an agent needs to call it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so the baseline of 4 applies. There is nothing for the description to disambiguate, and it correctly implies a parameterless lookup.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (returns/explains) and resource (the five verdicts access_check_url can give, their meanings, next moves, confidence bases). It explicitly positions itself as the companion to access_check_url, so an agent can distinguish it from both siblings without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a clear triggering context: 'for an agent that has just been refused' and 'the key to reading access_check_url results.' That routes usage well, though it never states when not to call it or names a competing alternative beyond the implicit pairing with access_check_url.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Returns the AI crawler tokens this index reads robots.txt rules for — only tokens their operators document — plus the widely-written tokens that no operator sends, which make a robots rule unenforceable, and the list of verdict names. Useful when writing or auditing a robots.txt.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnly, idempotent, non-destructive, closed-world). The description adds substantive domain context beyond that: it separates operator-documented tokens from widely-written tokens no operator sends, and explains that the latter make a robots rule unenforceable. It stops short of describing output format or ordering.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The primary return content is front-loaded in the opening clause, followed by the supporting categories and then a brief usage note. Despite the dense parenthetical, every clause carries information and nothing is redundant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no parameters and no output schema, the description must carry the burden of explaining what comes back, and it does: it names the token categories and the verdict-name list. An agent knows what to expect without a schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is nothing to document and the baseline of 4 applies. The description correctly spends no space on parameter semantics and instead describes the returned content.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Names a specific verb (returns) and resource (the AI crawler tokens this index reads robots.txt rules for), and enumerates the three categories of content returned. It does not explicitly contrast itself with access_check_url or access_explain_verdicts, though the scope is clear enough to distinguish it in practice.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit use context — 'Useful when writing or auditing a robots.txt' — which tells the agent when this tool is relevant. It does not state when NOT to use it or name an alternative sibling, so it falls short of the top band.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
access_check_url2 fields changedInput schema / properties / includeEvidence / descriptionPrevious value: -"Include every request made, with status and headers (default false: it is large)"New value: +"When true, the answer includes every request made, with status and headers (default false: it is large)"Input schema / properties / skipWellKnown / descriptionPrevious value: -"Skip the five well-known probes; faster, and they are almost always absent"New value: +"When true, the five well-known probes are skipped; faster, and those files are almost always absent"access_check_urlaccess_explain_verdictsaccess_list_agentsCheck which AI crawlers a site allows and see pages the way AI crawlers do. Free, read-only.
Ask before you fetch: will this domain serve your crawler, refuse it, or charge it?
Free check whether a website is technically open to AI assistants (AI crawlers, llms.txt, JSON-LD).
Free public web freshness and response-metadata checks for AI agents.
Add one secure layer between your agents and this server.
