Crawl a site's design system
crawl_siteCrawls a website's pages to extract shared design tokens (colors, fonts, CSS variables) and page structures, generating a site-wide design guide with per-page breakdowns.
Instructions
Crawl same-origin pages breadth-first from a start URL and merge each page's design analysis into a site-wide design system: colors, fonts, and CSS variables shared across pages versus unique to one page, plus a page inventory with section outlines. With outputDir, writes per-page design/layout JSON and screenshots plus a SITE.md guide naming the pages worth replicating individually.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Absolute http:// or https:// start URL; same-origin links are crawled from here. | |
| maxDepth | No | Maximum link depth from the start URL. | |
| maxPages | No | Maximum pages to load, start page and failed loads included. | |
| outputDir | No | Absolute directory: writes crawl/pages/<slug>/design.json + layout.json (+ viewport.jpg), crawl/site.json, and crawl/SITE.md. | |
| timeoutMs | No | Navigation timeout in milliseconds. | |
| userAgent | No | Optional browser user-agent override. | |
| waitUntil | No | Page lifecycle event to await before extraction. | domcontentloaded |
| autoScroll | No | Scroll through the page before extraction to trigger lazy-rendered content. | |
| maxScrolls | No | Maximum viewport-sized scroll steps when autoScroll is enabled. | |
| stripQuery | No | Treat URLs differing only by query string as the same page. | |
| viewportWidth | No | Browser viewport width in CSS pixels. | |
| viewportHeight | No | Browser viewport height in CSS pixels. | |
| excludePatterns | No | Glob patterns for paths to skip. Exclude wins over include. | |
| includePatterns | No | Glob patterns ('*' wildcard) a path must match to be crawled. Empty means all paths. | |
| maxLinksPerPage | No | Links harvested per page. | |
| includeScreenshots | No | With outputDir, save a viewport screenshot per page. | |
| maxElementsPerPage | No | DOM elements analyzed per page (lighter than analyze_site). | |
| allowPrivateNetwork | No | Allow localhost and private-network URLs. Enable only for trusted sites. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |