site_crawl
Crawl any website recursively to collect per-page SEO data including titles, meta descriptions, headings, and issues for site-wide audits and content analysis.
Instructions
Crawl a website and return data for every discovered page (up to max_pages).
Recursively follows internal links from the starting URL. Use for site-wide
audits, content inventory, internal link analysis, finding orphan pages,
detecting missing meta tags across a site, or any task needing full-site data.
Returns per-page SEO signals: title, meta description, headings, issues.
Args:
url: Starting URL (e.g. "https://example.com")
max_pages: Maximum pages to crawl (default: 50, max: 100)
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| config | No | ||
| max_pages | No |