scrape_url
Scrape any URL and retrieve content as markdown or HTML. Customize with tags, timeouts, and main-content-only extraction.
Instructions
Scrape a URL and return the content as markdown
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to scrape | |
| formats | No | Content formats to extract (default: ['markdown']) | |
| timeout | No | Maximum time in milliseconds to wait for the page to load | |
| waitFor | No | Time in milliseconds to wait for dynamic content to load | |
| excludeTags | No | HTML tags to exclude from extraction | |
| includeTags | No | HTML tags to specifically include in extraction | |
| onlyMainContent | No | Extract only the main content, filtering out navigation, footers, etc. |