Site crawl
crawl_siteCrawl a client site from an allowlisted start URL, respecting robots.txt, to audit broken links, redirects, duplicate titles and descriptions, and orphan pages.
Instructions
Rate-limited crawl from a start URL inside the client's allowlist, obeying robots.txt. Reports broken links, redirects, duplicate titles and descriptions, and orphan pages (sitemap URLs no crawled page links to).
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"client": "<client>",
"url": "<url>"
},
"name": "crawl_site"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Absolute URL inside the client's allowlist. | |
| client | Yes | Client id from the server's config (dental, cafe, plumber). | |
| maxPages | No | Default 50; capped by the server's MAX_CRAWL_PAGES. |