Fetches a public web page and returns its content as clean Markdown.
WHEN TO USE: you need to read a page — documentation, an article, a product page — to answer, summarize or quote it; or the user wants a page as Markdown.
HOW IT WORKS:
- url: a public http(s) address. Private networks, localhost and addresses with credentials are refused.
- Reads the HTML the server sends (like a simple browser, without running JavaScript). Pages that build their content with JavaScript have no text to convert — the error says so.
- onlyMainContent (default true) drops menus, headers, footers, sidebars, cookie banners and ads. Set false to keep the whole page.
- pages above 1 also follows links on the same site (same folder and below), breadth-first, up to that many pages (max 20), respecting robots.txt. Slower: a few seconds per page.
- Limits per page: 15 seconds and 5 MB. PDFs and non-HTML files are not converted.
RETURNS: for each page, its Markdown with absolute links and images, plus title, description and language when the page has them; pages that failed are listed with the reason. link: this tool's web page with the same request.
ERRORS say why the page could not be read and whether retrying makes sense.