docs_add_url
Add documentation from a URL by scraping a page, optionally crawling linked pages and rendering JavaScript-built sites, so content can be stored and searched locally.
Instructions
Add documentation from a URL. Features: recursive crawling (follows links), Playwright support for JS-rendered sites (React, Vue, etc.), smart content detection. Use recursive=true to crawl multiple pages.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"url": "<url>"
},
"name": "docs_add_url"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to scrape | |
| name | No | Optional: display name for the source | |
| maxDepth | No | Maximum depth for recursive crawling (default: 2) | |
| maxPages | No | Maximum pages to crawl when recursive is true (default: 20) | |
| sourceId | No | Optional: custom source ID (auto-generated from URL if not provided) | |
| recursive | No | Recursively crawl linked pages | |
| urlPattern | No | Optional: regex pattern to filter URLs during recursive crawl | |
| description | No | Optional: description for the source | |
| usePlaywright | No | Use Playwright for JS-rendered pages (slower but handles React/Vue/Angular sites) |