crawl_site_markdown
Crawl a website from a single URL and convert each page into its own Markdown document for docs sites, blogs, or knowledge bases. Set maxPages to control cost.
Instructions
Crawl a website starting from one URL and return one Markdown document per page (docs sites, blogs, knowledge bases). Billed per page crawled, so keep maxPages as low as the task allows.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The page to start crawling from, e.g. https://docs.apify.com/platform. | |
| maxDepth | No | How many links deep to follow from the start URL. | |
| maxPages | No | Maximum number of pages to crawl and convert. This is the billed unit — raise only as needed. | |
| useSitemap | No | Also seed the crawl from the domain's sitemap.xml. | |
| outputFormat | No | Content format to return per page. | markdown |
| sameDomainOnly | No | Only follow links on the same domain as the start URL. |