Fetch a single webpage and extract its main content as clean markdown.

USE THIS TOOL WHEN:

• You need to read an article, documentation page, or blog post

• You want clean, readable text without boilerplate (navbars, ads, footers)

• The user provides a specific URL to read

DO NOT USE WHEN:

• You need to fetch multiple URLs (use fetch_batch instead - fewer round trips)

• You want to discover what pages exist on a site (use discover_links first)

PAGINATION: Large pages are automatically truncated. The response will include 'use start_index=N to continue' - call again with that value to get more content.

EXAMPLES:

• fetch(url="https://docs.example.com/guide") → Get documentation page

• fetch(url="https://example.com/api", max_length=2000) → Shorter response to save context

• fetch(url="https://example.com/data", include_tables=True) → Preserve tabular data

Fetch multiple webpages in a single request and return combined content.

USE THIS TOOL WHEN:

• You have 2-10 URLs to read (e.g., from discover_links results)

• Comparing content across multiple pages

• Gathering context from several documentation pages at once

KEY BENEFIT: One tool call instead of multiple fetch() calls = fewer round trips, faster results, and reduced overhead in supervised/approval workflows.

WORKFLOW EXAMPLE:

1. discover_links(url="https://docs.example.com", filter_pattern="/api/")

2. fetch_batch(urls=[...returned links...], max_length_per_url=1500)

NOTES:

• Each URL's content is separated by '---' dividers

• Failed URLs show inline errors without stopping other fetches

• Robots.txt is NOT checked for batch fetches (assumes prior discovery)

Discover all links on a webpage. Use this BEFORE fetch_batch to find relevant URLs.

USE THIS TOOL WHEN:

• Exploring a documentation site to find relevant pages

• Building a list of URLs to fetch in batch

• Finding all subpages under a section (e.g., all /api/ docs)

• Checking what content exists before deciding what to read

RECOMMENDED WORKFLOW:

1. discover_links(url="https://docs.example.com/", filter_pattern="/guide/")

2. Review the returned links and select relevant ones

3. fetch_batch(urls=[selected_urls], max_length_per_url=1500)

FILTER EXAMPLES:

• filter_pattern="/docs/" → Only links containing '/docs/'

• filter_pattern="getting-started|quickstart" → Links with either term

• filter_pattern=".md$" → Only markdown file links

NOTES:

• Returns up to 100 links (more are noted but omitted to save context)

• Relative URLs are automatically resolved to absolute URLs

• JavaScript/mailto/anchor links are excluded

Fetch and parse an llms.txt file to discover LLM-friendly documentation.

USE THIS TOOL WHEN:

• A site provides an llms.txt file for AI-friendly content discovery

• You need to understand what documentation is available

• You want to fetch structured docs in a single request

WHAT IS llms.txt: A proposal for sites to provide LLM-friendly content at /llms.txt. It's a markdown file listing documentation links with descriptions. See https://llmstxt.org for the specification.

WORKFLOW:

1. fetch_llms_txt(url="https://example.com/llms.txt") → Get structure

2. Review sections and links

3. Either use include_content=True or fetch_batch for specific pages

EXAMPLES:

• fetch_llms_txt(url="https://fastht.ml/docs/llms.txt")

• fetch_llms_txt(url="https://example.com/llms.txt", include_content=True)