ai_scraper
Scrape web pages into Markdown, JSON, or CSV, with JavaScript rendering and schema-based structured data extraction for complex sites.
Instructions
Scrape the contents of the web page and return the data in the specified format.
Schema is required only if output_format is json or csv. 'render_javascript' is used to render javascript heavy websites.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to scrape | |
| schema | No | The JSON schema to use for structured data extraction from the scraped page. Only required if output_format is json, csv or toon. | |
| geo_location | No | Two letter ISO country code to use for the scrape proxy. | |
| output_format | No | The format of the output. If json, csv or toon, the schema is required. Markdown returns full text of the page. CSV returns data in CSV format, tabular like data. Toon(Token-Oriented Object Notation) returns data in Toon format, which is optimized for AI agents. | markdown |
| render_javascript | No | Whether to render the HTML of the page using javascript. Much slower, therefore use it only for websites that require javascript to render the page.Unless user asks to use it, first try to scrape the page without it. If results are unsatisfactory, try to use it. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |