spider_ai_crawl
Crawl websites guided by natural language prompts. Specify the content to find, and the AI directs the crawl to extract relevant data.
Instructions
AI-guided crawling using natural language prompts. Describe what content to find and Spider's AI will guide the crawl. REQUIRES an active AI subscription plan (https://spider.cloud/ai/pricing).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to crawl | |
| limit | No | Maximum pages to crawl | |
| prompt | Yes | Natural language prompt to guide the crawl (e.g. 'Find all product pages and extract pricing') | |
| cookies | No | HTTP cookies | |
| request | No | Request type | |
| proxy_enabled | No | Enable premium proxies | |
| return_format | No | Output format |