Web scrape
web_scrapeRetrieve main text from up to 10 URLs in parallel, converting pages to markdown with individual truncation limits. Batch all links into a single call.
Instructions
Downloads pages by their links and returns the main text as markdown. Links are fetched in parallel; a single call handles up to 10 of them, so batch all links you need into one call instead of calling the tool per link.
Size: the 'max_chars' limit applies to each page separately, not to the whole response — 5 links stay under the limit each and the response holds all 5 texts. The limit counts characters, not tokens. A page cut short ends with '[truncated]' and its 'truncated' field is set; the cut-off part cannot be fetched afterwards, so treat what you got as all there is for that page.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| urls | Yes | links to pages, downloaded in parallel | |
| max_chars | No | truncate each page separately to N characters, default and maximum 20000; only lowers the limit, higher values are ignored | |
| robots_txt | No | respect the page robots.txt | |
| timeout_ms | No | timeout for the whole call in milliseconds | |
| remove_links | No | strip markdown links from the text |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| results | Yes | ||
| metadata | Yes |