webscraping_ai_fields
Extract specified data fields from any web page by providing a URL and a dictionary of field names with natural language extraction instructions.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL of the target page. | |
| fields | Yes | Dictionary of field names with instructions for extraction. | |
| timeout | No | Maximum web page retrieval time in ms (20000 by default, maximum is 30000). | |
| js | No | Execute on-page JavaScript using a headless browser (false by default). | |
| js_timeout | No | Maximum JavaScript rendering time in ms (3000 by default). | |
| wait_for | No | CSS selector to wait for before returning the page content. | |
| proxy | No | Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page. | datacenter |
| country | No | Country of the proxy to use (US by default). | |
| custom_proxy | No | Your own proxy URL in "http://user:password@host:port" format. | |
| device | No | Type of device emulation. | |
| error_on_404 | No | Return error on 404 HTTP status on the target page (false by default). | |
| error_on_redirect | No | Return error on redirect on the target page (false by default). | |
| js_script | No | Custom JavaScript code to execute on the target page. |
Implementation Reference
- src/index.js:255-263 (handler)The handler function for the 'webscraping_ai_fields' tool. It calls client.fields() with the url and fields dictionary, then returns the result as a formatted JSON string via createSanitizedResponse.
async ({ url, fields, ...options }) => { try { const result = await client.fields(url, fields, options); return createSanitizedResponse(JSON.stringify(result, null, 2), url); } catch (error) { return createSanitizedResponse(error.message, url, true); } } ); - src/index.js:250-254 (schema)Schema definition for the 'webscraping_ai_fields' tool: url (string), fields (record of string->string for field name to extraction instruction), and common options (timeout, js, proxy, etc.).
{ url: z.string().describe('URL of the target page.'), fields: z.record(z.string()).describe('Dictionary of field names with instructions for extraction.'), ...commonOptionsSchema }, - src/index.js:248-263 (registration)Registration of the 'webscraping_ai_fields' tool via server.tool() on the McpServer instance.
server.tool( 'webscraping_ai_fields', { url: z.string().describe('URL of the target page.'), fields: z.record(z.string()).describe('Dictionary of field names with instructions for extraction.'), ...commonOptionsSchema }, async ({ url, fields, ...options }) => { try { const result = await client.fields(url, fields, options); return createSanitizedResponse(JSON.stringify(result, null, 2), url); } catch (error) { return createSanitizedResponse(error.message, url, true); } } ); - src/index.js:82-88 (helper)The client.fields() helper method that sends a request to the /ai/fields endpoint, serializing the fields dictionary as JSON.
async fields(url, fields, options = {}) { return this.request('/ai/fields', { url, fields: JSON.stringify(fields), ...options }); }