search_engine
Scrape search results from Google, Bing, or Yandex to extract SERP data including URLs, titles, and descriptions in markdown format for web research and data collection.
Instructions
Scrape search results from Google, Bing or Yandex. Returns SERP results in markdown (URL, title, description)
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| cursor | No | Pagination cursor for next page | |
| engine | No | ||
| query | Yes |
Implementation Reference
- server.js:143-158 (handler)The core handler logic wrapped in tool_fn, which sends a POST request to BrightData API using the search URL generated by search_url helper.execute: tool_fn('search_engine', async({query, engine, cursor})=>{ let response = await axios({ url: 'https://api.brightdata.com/request', method: 'POST', data: { url: search_url(engine, query, cursor), zone: unlocker_zone, format: 'raw', data_format: 'markdown', }, headers: api_headers(), responseType: 'text', }); return response.data; }),
- server.js:134-142 (schema)Zod schema defining the input parameters: query (required), engine (optional, default google), cursor (optional).parameters: z.object({ query: z.string(), engine: z.enum([ 'google', 'bing', 'yandex', ]).optional().default('google'), cursor: z.string().optional().describe('Pagination cursor for next page'), }),
- server.js:130-159 (registration)The addTool call registering the 'search_engine' tool with FastMCP, including name, description, parameters, and execute.addTool({ name: 'search_engine', description: 'Scrape search results from Google, Bing or Yandex. Returns ' +'SERP results in markdown (URL, title, description)', parameters: z.object({ query: z.string(), engine: z.enum([ 'google', 'bing', 'yandex', ]).optional().default('google'), cursor: z.string().optional().describe('Pagination cursor for next page'), }), execute: tool_fn('search_engine', async({query, engine, cursor})=>{ let response = await axios({ url: 'https://api.brightdata.com/request', method: 'POST', data: { url: search_url(engine, query, cursor), zone: unlocker_zone, format: 'raw', data_format: 'markdown', }, headers: api_headers(), responseType: 'text', }); return response.data; }), });
- server.js:780-789 (helper)Helper function to construct the search engine URL based on engine, query, and cursor/page.function search_url(engine, query, cursor){ let q = encodeURIComponent(query); let page = cursor ? parseInt(cursor) : 0; let start = page * 10; if (engine=='yandex') return `https://yandex.com/search/?text=${q}&p=${page}`; if (engine=='bing') return `https://www.bing.com/search?q=${q}&first=${start + 1}`; return `https://www.google.com/search?q=${q}&start=${start}`; }