crawl_website
Crawl a website and extract structured content from all accessible pages. Returns page titles, meta descriptions, headings, body text, internal/external links, and page structure. Use for SEO audits, content inventory, site mapping, or data extraction for analysis.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Website URL to crawl (e.g. 'https://www.example.com', 'example.com') | |
| max_pages | No | Maximum number of pages to crawl (default 10, higher for full site scans) |