List AI crawlers in the registry
list_crawlersList AI crawlers from the registry, optionally filtered by operator (OpenAI, Anthropic, Google), purpose (training, search-index, user-triggered, search-and-training, other), or status (active, retired, renamed). Use it to compare how operators split training from search agents, or to find every training crawler before writing robots.txt rules. Results are paginated. Use when: Use when someone wants to browse AI crawlers by operator, purpose, or status. Do not use when: Do not use when someone names one exact token; look that token up directly.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Maximum records to return in this call. | |
| query | No | Free-text search over token, operator, and notes. | |
| offset | No | Number of matching records to skip. Use next_offset to page forward. | |
| status | No | Limit to one lifecycle status. | |
| purpose | No | Limit to one purpose class. | |
| operator | No | Limit to one operator, matched exactly but case-insensitively, for example 'OpenAI'. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| results | Yes | Matching crawler records (summary fields; call lookup_crawler for the full record). | |
| has_more | Yes | True when more results exist beyond this page. | |
| truncated | Yes | True when results were shortened to fit the 50 KB output cap. Use the paginated REST API for the full set. | |
| next_offset | Yes | Pass as 'offset' in the next call to continue paging. Null when has_more is false. | |
| total_count | Yes | Total number of matches across all pages. | |
| corpus_version | Yes | Semantic version of the movahedi.ca statutory corpus this answer was drawn from, for example v2026.12.0. Pin it when citing; when it changes, re-read the affected resources and the change log at https://movahedi.ca/changelog/. |