docs_add_url
Add documentation from a URL by scraping a page, optionally crawling linked pages and rendering JavaScript-built sites, so content can be stored and searched locally.
Instructions
Add documentation from a URL. Features: recursive crawling (follows links), Playwright support for JS-rendered sites (React, Vue, etc.), smart content detection. Use recursive=true to crawl multiple pages.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to scrape | |
| name | No | Optional: display name for the source | |
| maxDepth | No | Maximum depth for recursive crawling (default: 2) | |
| maxPages | No | Maximum pages to crawl when recursive is true (default: 20) | |
| sourceId | No | Optional: custom source ID (auto-generated from URL if not provided) | |
| recursive | No | Recursively crawl linked pages | |
| urlPattern | No | Optional: regex pattern to filter URLs during recursive crawl | |
| description | No | Optional: description for the source | |
| usePlaywright | No | Use Playwright for JS-rendered pages (slower but handles React/Vue/Angular sites) |