Bintu Scrape Text
bintu_scrape_textToken-Sanitized Context Gateway for AI agents: fetches one public absolute http(s) URL (1 MiB cap, HTML/XHTML only, nav/ads/footer stripped, article/main preferred) and returns readable text capped at 6,000 characters. Default mode=scrape returns the token-metered envelope. Use mode=chunk (alias chunked) to split text into max_tokens-sized chunks ready for vector databases; use mode=summarize to truncate to a token budget. Every call returns tokens, chunk_count, cache_hit, tokens_saved_estimate for predictable context budgets. truncated=true means content was actually lost (the 6,000-character cap, or summarize dropping sentences); split=true only means the text was divided across several chunks, which never loses anything. tokens is an approximate whitespace-token count (about one per word), not a real tokenizer, so treat it as a sizing estimate rather than an exact billing figure; tokens_saved_estimate is nonzero only in summarize mode. JavaScript rendering is disabled unless enabled — render_js_disabled means retry with render_js=false. Fails cleanly with invalid_or_unsafe_url, render_js_disabled, rate_limited — all in-band so the model can self-correct.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Absolute public HTTP or HTTPS URL to extract. Private IPs, non-http schemes, and unsafe targets fail with invalid_or_unsafe_url. | |
| mode | No | Extraction mode: scrape returns the metered envelope; chunk splits text into max_tokens-sized chunks for vector databases (pitch alias: chunked); summarize truncates to a token budget. | scrape |
| cache | No | Use the in-memory + Supabase cache when available. | |
| render_js | No | Render the page in Chromium before extracting text. Usually disabled on free infra; when off the call fails with render_js_disabled — retry with false. | |
| max_tokens | No | Token budget for chunk/summarize modes (ignored in scrape mode). Pitch alias: chunk_token_budget. Note: extracted text is always capped at 6,000 characters first, so that cap usually binds before this budget on long pages. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| content | Yes | MCP content blocks. content[0].text is always the unified JSON envelope: {mode, tokens, chunk_count, cache_hit, source_url, extracted_at, tokens_saved_estimate, truncated, chunks:[{index, text, tokens}]} for predictable context budgets in every mode. | |
| isError | No | True when extraction failed; content[0].text holds a code-prefixed hint (e.g. invalid_or_unsafe_url, render_js_disabled). |