docs_add_sitemap
Crawl an entire documentation site from its sitemap.xml, convert all pages to Markdown, and store locally to add complete doc sites.
Instructions
Crawl an entire documentation site from its sitemap.xml. Scrapes all pages (up to maxPages), converts to Markdown, and stores locally. Use for adding complete doc sites.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | Optional: display name for the source | |
| maxPages | No | Maximum number of pages to crawl (default: 50, max: 200) | |
| sourceId | No | Optional: custom source ID (auto-generated from URL if not provided) | |
| sitemapUrl | Yes | The sitemap URL (e.g., "https://docs.example.com/sitemap.xml") | |
| urlPattern | No | Optional: regex pattern to filter URLs (e.g., "/docs/" to only include URLs containing /docs/) | |
| description | No | Optional: description for the source |