scrape
Extract text content from any URL for data analysis, research, or content processing using web scraping functionality.
Instructions
Scrape and extract text content from a URL ($0.001)
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes |
Implementation Reference
- index.js:28-28 (registration)The 'scrape' tool is defined in the TOOLS array with its schema, endpoint, and pricing.
{ name: 'scrape', description: 'Scrape and extract text content from a URL', inputSchema: { type: 'object', properties: { url: { type: 'string' } }, required: ['url'] }, endpoint: '/scrape', price: '$0.001' }, - index.js:50-79 (handler)The callTool function serves as the generic handler that executes the tool logic by fetching the configured endpoint.
async function callTool(endpoint, params) { const fetch = (await import('node-fetch')).default; const isGet = ['GET'].includes((TOOLS.find(t => t.endpoint === endpoint) || {}).method); const url = isGet ? `${BASE_URL}${endpoint}?${new URLSearchParams(params)}` : `${BASE_URL}${endpoint}`; const res = await fetch(url, { method: isGet ? 'GET' : 'POST', headers: { 'Content-Type': 'application/json', 'Authorization': `Bearer ${API_KEY}`, }, body: isGet ? undefined : JSON.stringify(params), }); const text = await res.text(); let data; try { data = JSON.parse(text); } catch { data = { raw: text }; } if (!res.ok) { if (res.status === 402) { throw new Error(`Insufficient credits. Add credits at https://iteratools.com. Cost: ${TOOLS.find(t=>t.endpoint===endpoint)?.price || 'see docs'}`); } throw new Error(`API error ${res.status}: ${text.substring(0, 200)}`); } return data; } - index.js:94-115 (handler)The CallToolRequest handler routes the request to the callTool function based on the tool name.
server.setRequestHandler(CallToolRequestSchema, async (request) => { const { name, arguments: args } = request.params; if (!API_KEY) { return { content: [{ type: 'text', text: 'Error: ITERATOOLS_API_KEY environment variable not set. Get a key at https://iteratools.com' }], isError: true, }; } const tool = TOOLS.find(t => t.name === name); if (!tool) { return { content: [{ type: 'text', text: `Unknown tool: ${name}` }], isError: true }; } try { const result = await callTool(tool.endpoint, args); return { content: [{ type: 'text', text: JSON.stringify(result, null, 2) }] }; } catch (err) { return { content: [{ type: 'text', text: `Error: ${err.message}` }], isError: true }; } });