Whether an AI agent may fetch a URL, on what terms, and at what price. Free, no key, no signup.
Scored across 3 tools
Each tool has a clearly distinct purpose: checking a URL's access terms, explaining the verdict taxonomy, and listing AI crawler tokens. There is no overlap or ambiguity between the three tools.
All tool names follow a consistent access_ + verb + noun snake_case pattern: access_check_url, access_explain_verdicts, access_list_agents. The convention is predictable and readable.
Three tools are well-scoped for a focused lookup service; each tool performs a necessary and non-redundant function. The count is at the low end but appropriate for the domain.
The core read-only workflow (check a URL, explain verdicts, list agents) is covered without dead ends. Minor gaps exist, such as no batch-checking or browsing of indexed origins, but agents can work around these with repeated calls.
Says whether an AI agent may fetch a URL, on what terms, and at what price. Reads robots.txt for a generic agent and for every AI user-agent the operators document, the Content-signal and Content-Usage directives, any RSL licence the origin links, and the live response to an honestly-identified request.
Verdicts: allowed (it served us), allowed_if_identified (the written policy permits
this but bot management refused an anonymous request — signing as a named crawler is the
documented route), charged (there is a price, quoted where the rail will show one),
blocked (the site disallows the agents that would want this, with nothing to buy), and
unknown (the origin did not answer, or the URL is not there).
Never evades: no CAPTCHA solving, no proxies, no spoofed user-agents. Refusals come back as findings with the evidence attached.
Inputs: url (required), skipWellKnown, includeEvidence.
Returns the five verdicts access_check_url can give, what each one means for an agent that has just been refused, and the next move for each, plus what each confidence level rests on. Free, makes no network request, and returns the same answer every time: it is the key to reading access_check_url results.
Inputs: no named parameters.
Returns the AI crawler tokens this index reads robots.txt rules for — only tokens their operators document — plus the widely-written tokens that no operator sends, which make a robots rule unenforceable, and the list of verdict names. Useful when writing or auditing a robots.txt.
Inputs: no named parameters.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Check which AI crawlers a site allows and see pages the way AI crawlers do. Free, read-only.
Ask before you scrape: is a site open, may I fetch this URL, and how do I extract a page. No key.
Ask before you fetch: will this domain serve your crawler, refuse it, or charge it?
Free check whether a website is technically open to AI assistants (AI crawlers, llms.txt, JSON-LD).
Add one secure layer between your agents and this server.
