scrape_domain
Extract data from a specified domain by scraping up to a defined number of pages. Enables SEO analysis, keyword research, and SERP data collection via FetchSERP MCP Server.
Instructions
Scrape a domain
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| domain | Yes | The domain to scrape | |
| max_pages | No | The maximum number of pages to scrape (up to 200). Default: 10 |
Input Schema (JSON Schema)
{
"properties": {
"domain": {
"description": "The domain to scrape",
"type": "string"
},
"max_pages": {
"default": 10,
"description": "The maximum number of pages to scrape (up to 200). Default: 10",
"maximum": 200,
"type": "integer"
}
},
"required": [
"domain"
],
"type": "object"
}
Implementation Reference
- index.js:647-648 (handler)Handler case for the scrape_domain tool within handleToolCall method. Proxies the request to the external FetchSERP API endpoint '/api/v1/scrape_domain'.case 'scrape_domain': return await this.makeRequest('/api/v1/scrape_domain', 'GET', args, null, token);
- index.js:264-282 (schema)Tool schema definition including name, description, and input schema for 'scrape_domain' in the list of tools returned by ListToolsRequestSchema handler.{ name: 'scrape_domain', description: 'Scrape a domain', inputSchema: { type: 'object', properties: { domain: { type: 'string', description: 'The domain to scrape', }, max_pages: { type: 'integer', description: 'The maximum number of pages to scrape (up to 200). Default: 10', default: 10, maximum: 200, }, }, required: ['domain'], },
- index.js:565-613 (helper)Shared helper method used by all tool handlers to make authenticated HTTP requests to the FetchSERP API.async makeRequest(endpoint, method = 'GET', params = {}, body = null, token = null) { const fetchserpToken = token || process.env.FETCHSERP_API_TOKEN; if (!fetchserpToken) { throw new McpError( ErrorCode.InvalidRequest, 'FETCHSERP_API_TOKEN is required' ); } const url = new URL(`${API_BASE_URL}${endpoint}`); // Add query parameters for GET requests if (method === 'GET' && Object.keys(params).length > 0) { Object.entries(params).forEach(([key, value]) => { if (value !== undefined && value !== null) { if (Array.isArray(value)) { value.forEach(v => url.searchParams.append(`${key}[]`, v)); } else { url.searchParams.append(key, value.toString()); } } }); } const fetchOptions = { method, headers: { 'Authorization': `Bearer ${fetchserpToken}`, 'Content-Type': 'application/json', }, }; if (body && method !== 'GET') { fetchOptions.body = JSON.stringify(body); } const response = await fetch(url.toString(), fetchOptions); if (!response.ok) { const errorText = await response.text(); throw new McpError( ErrorCode.InternalError, `API request failed: ${response.status} ${response.statusText} - ${errorText}` ); } return await response.json(); }