spider_ai_crawl
Describe your crawl goals in plain English; the AI optimizes the crawl to extract relevant data from any URL.
Instructions
AI-guided website crawling. Describe what you want in plain English and Spider's AI optimizes the crawl automatically. Requires an active AI subscription (https://spider.cloud/ai/pricing).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to crawl | |
| limit | No | Max pages to crawl | |
| prompt | Yes | Natural language instructions (e.g. 'Find all product pages and extract pricing info') | |
| cookies | No | HTTP cookies | |
| request | No | Request type: http (fast), chrome (JS rendering), smart (auto-detect). Default: smart | |
| proxy_enabled | No | Enable premium proxies | |
| return_format | No | Output format. Default: raw |