Read one web page
telos.crawl.fetchRetrieve text, feed items, sitemap URLs, or JSON from one public URL read-only, checking robots.txt first and returning a fetch receipt.
Instructions
Use when an agent needs the text, feed items, sitemap URLs or JSON of one public URL. Read-only and zero-auth; checks robots.txt first and stops at a bot check. Reaches the network: one honest Telos User-Agent, per-host pacing with backoff on 429 and 503, no login, no cookies. Returns JSON with the parsed page and a receipt of every fetch.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | http or https URL |