Fetch a web page as Markdown
fetch_pageRead one public web page and return its main content as clean Markdown: headings, paragraphs, lists, quotes, code blocks, tables and absolute links, without menus, footers, scripts or cookie banners. Also returns the title, description, language, canonical address and up to 100 links from the content. robots.txt is checked first and a disallowed page is skipped with the reason. It never signs in, passes a paywall, solves a CAPTCHA or runs page scripts. Pass max_chars for more than 20000 characters. Not for what is said in a video: a YouTube video's transcript comes from transcribe_media in Transcript Kit.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Full address of one public web page, such as "https://example.com/docs/intro". http is upgraded to https. | |
| links | No | Set false to leave out the link list. Default true: up to 100 distinct links from the main content. | |
| max_chars | No | Longest Markdown to return, cut at a block end. Default 20000, from 1000 to 100000: a value outside that range is moved into it, with a note. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| lang | No | ||
| note | Yes | ||
| as_of | Yes | ISO 8601 time the data was published by its source or fetched from it. A cached answer keeps the time of its fetch. | |
| chars | No | ||
| links | No | ||
| notes | No | ||
| title | No | ||
| words | No | ||
| reason | No | ||
| source | Yes | Who the data comes from, to cite and to check it against. | |
| status | Yes | ||
| message | No | ||
| markdown | No | ||
| canonical | No | ||
| final_url | No | ||
| truncated | No | ||
| fetched_at | No | ||
| from_cache | No | ||
| description | No | ||
| links_total | No | ||
| content_type | No | ||
| untrusted_source | Yes | ||
| retry_after_seconds | No |