Fetch a single web page and return its content as Markdown.

This is the primary tool for reading web content. Use this when you know which URL contains the information you need.

Best for: Reading documentation pages, articles, blog posts, API references, or any single page where you know the URL.

Not recommended for: When you don't know which page has the info (use map_url first), or when you need content from many pages (use crawl_url).

Args: url: The URL to fetch. render: If true (default), renders JavaScript with a headless browser. Set to false for faster fetch of static pages.

Discover URLs on a website without fetching full page content.

Returns a list of URLs found by crawling from the starting URL. Use this to find the right page before scraping it.

Best for: Finding documentation pages, locating specific content on a site, understanding site structure before scraping.

Typical workflow: map_url to find URLs -> scrape_url on the right page.

Args: url: The starting URL to discover links from. limit: Maximum number of URLs to discover (default: 50, max: 100000). depth: How many links deep to follow (default: 2). include_subdomains: If true, follows links to subdomains. include_external_links: If true, follows links to external domains. include_patterns: Only visit URLs matching these wildcard patterns (e.g. "https://example.com/docs/**"). exclude_patterns: Skip URLs matching these wildcard patterns.

Crawl multiple pages starting from a URL and return all content as Markdown.

Best for: Fetching entire documentation sections, blog archives, or multiple related pages at once.

Not recommended for: Single pages (use scrape_url — it's faster). Large sites without filters (responses can be very large and exceed token limits).

Tip: Use include_patterns to scope the crawl (e.g. "https://example.com/docs/**").

Args: url: The starting URL to crawl. limit: Maximum number of pages to crawl (default: 10, max: 100000). depth: Maximum link depth from the starting URL (default: 1). include_subdomains: If true, follows links to subdomains. include_external_links: If true, follows links to external domains. include_patterns: Only visit URLs matching these wildcard patterns. exclude_patterns: Skip URLs matching these wildcard patterns. render: If true (default), renders JavaScript. Set false for faster static fetch.