Fetch Page Content
infobroker_fetch_pageFetch a URL to extract clean readable text, ranked passages for a question, or the page's last-updated date.
Instructions
Fetch a URL and extract clean content via a renderer (Jina Reader by default; native-HTTP, Wikipedia, Internet Archive, arXiv, and Stack Exchange alternatives). Use when you have a URL and need readable text, passages ranked against a question, or the page's last-updated date. Do NOT use for topic search (use infobroker_search_web) or cross-source claim verification (use infobroker_verify_claims). question returns passages ranked against it, sized by passage_size and capped by max_passages; crawl bounds the same-origin crawl to config caps; extract adds JSON-LD, OpenGraph, and microdata; renderer selects the backend, and native_fetch is the keyless fallback when Jina is throttled or anti-bot challenged. max_length truncates the returned text only; a truncated page is written in full to a temp file. Makes external HTTP calls and needs no API key. Fetched pages auto-index into the knowledge base unless the content policy flags them (see infobroker_manage_kb): flag mode returns without storage, block mode refuses. Unreachable URLs return an [ERROR] envelope with remediation; success returns an [OK] envelope with status, provider, results, and meta.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to fetch: a single URL, or up to five URLs fetched in parallel | |
| crawl | No | Bounded same-origin crawl: recursively fetch same-origin pages up to config caps (default off) | |
| extract | No | Return structured metadata (JSON-LD, OpenGraph, microdata) alongside the content (default off) | |
| question | No | Question to extract ranked passages for, instead of returning the whole page | |
| renderer | No | Renderer: jina (default), native_fetch, wikipedia, internet_archive, arxiv, or stack_exchange | |
| max_length | No | Maximum characters to return (default 50000) | |
| detect_date | No | Detect and report the page's last-updated date (default from config) | |
| max_passages | No | Number of passages to return (default from config) | |
| passage_size | No | Target words per passage (default from config) |