search_engine
Extract search engine results from Google, Bing, or Yandex to obtain URLs, titles, and descriptions for web research and data collection.
Instructions
Scrape search results from Google, Bing or Yandex. Returns SERP results in markdown (URL, title, description)
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | ||
| engine | No | ||
| cursor | No | Pagination cursor for next page |
Implementation Reference
- server.js:143-158 (handler)Handler for the 'search_engine' tool. Constructs a search URL using the search_url helper, sends a POST request to BrightData API's /request endpoint using the unlocker zone, requests markdown format, and returns the scraped SERP results.
execute: tool_fn('search_engine', async({query, engine, cursor})=>{ let response = await axios({ url: 'https://api.brightdata.com/request', method: 'POST', data: { url: search_url(engine, query, cursor), zone: unlocker_zone, format: 'raw', data_format: 'markdown', }, headers: api_headers(), responseType: 'text', }); return response.data; }), - server.js:134-142 (schema)Zod schema defining the input parameters for the 'search_engine' tool: 'query' (required string), 'engine' (optional enum ['google', 'bing', 'yandex'], defaults to 'google'), 'cursor' (optional string for pagination).
parameters: z.object({ query: z.string(), engine: z.enum([ 'google', 'bing', 'yandex', ]).optional().default('google'), cursor: z.string().optional().describe('Pagination cursor for next page'), }), - server.js:130-159 (registration)Registers the 'search_engine' tool with the FastMCP server using addTool, specifying name, description, input schema, and execute handler.
addTool({ name: 'search_engine', description: 'Scrape search results from Google, Bing or Yandex. Returns ' +'SERP results in markdown (URL, title, description)', parameters: z.object({ query: z.string(), engine: z.enum([ 'google', 'bing', 'yandex', ]).optional().default('google'), cursor: z.string().optional().describe('Pagination cursor for next page'), }), execute: tool_fn('search_engine', async({query, engine, cursor})=>{ let response = await axios({ url: 'https://api.brightdata.com/request', method: 'POST', data: { url: search_url(engine, query, cursor), zone: unlocker_zone, format: 'raw', data_format: 'markdown', }, headers: api_headers(), responseType: 'text', }); return response.data; }), }); - server.js:780-789 (helper)Helper function that constructs the search engine URL (Google, Bing, or Yandex) based on the query, engine, and cursor (for pagination), properly encoding the query and calculating start/page parameters.
function search_url(engine, query, cursor){ let q = encodeURIComponent(query); let page = cursor ? parseInt(cursor) : 0; let start = page * 10; if (engine=='yandex') return `https://yandex.com/search/?text=${q}&p=${page}`; if (engine=='bing') return `https://www.bing.com/search?q=${q}&first=${start + 1}`; return `https://www.google.com/search?q=${q}&start=${start}`; }