Check whether an agent may fetch a URL
access_check_urlGiven a URL, say whether an AI agent may fetch it, on what terms, and at what price. Reads robots.txt for a generic agent and for every AI user-agent the operators document, the Content-signal and Content-Usage directives, any RSL licence the origin links, and the live response to an honestly-identified request.
Verdicts: allowed (it served us), allowed_if_identified (the written policy permits
this but bot management refused an anonymous request — signing as a named crawler is the
documented route), charged (there is a price, quoted where the rail will show one),
blocked (the site disallows the agents that would want this, with nothing to buy), and
unknown (the origin did not answer, or the URL is not there).
Never evades: no CAPTCHA solving, no proxies, no spoofed user-agents. Refusals come back as findings with the evidence attached.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Absolute http(s) URL of the page to check, e.g. https://example.com/article | |
| skipWellKnown | No | Skip the five well-known probes; faster, and they are almost always absent | |
| includeEvidence | No | Include every request made, with status and headers (default false: it is large) |