agent_profile
What this census measures and publishes about one AI crawler: how often it is disallowed in robots.txt, how often live requests carrying its user agent are refused at the network edge whatever robots.txt says, whether its operator documents it as honouring robots.txt, and where to correct any of that. Intended for the operator of the agent as much as for anyone studying it, so it includes the correction channel and the public page a claim can be disputed against.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| agent | Yes | Crawler token, e.g. gptbot, claudebot, ccbot, google-extended. |