Enables performing structured DuckDuckGo searches via the SERP API, returning results such as titles, URLs, and snippets.
Enables performing structured Google searches via the SERP API, returning results such as titles, URLs, and snippets.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@Bright Data MCP Server search Google for best laptops 2026"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
# Scout MCP
Free-first web access for AI agents. Bright Data only when a site actually blocks you.
Scout is a Model Context Protocol server that gives your AI agent a full browser and scraping toolkit. Most pages are fetched for free with a plain HTTP request. The paid Bright Data tier only kicks in when an anti-bot wall (Cloudflare, Akamai, CAPTCHA, etc.) is detected — which means most research workflows cost nothing.
No API key required to get started. Free tools work out of the box. Configure Bright Data only when you need it.
Every fetch follows a cost ladder and stops at the first rung that works:
1. Free direct fetch → realistic browser headers, no cost
2. Bright Data → paid proxy + unlocker, only on detected blockAn in-memory skip-list (persisted to disk) remembers which domains blocked the free tier so repeat calls go straight to Bright Data — no wasted attempt. Skip-list entries expire after 30 minutes so the server re-probes in case a site drops its protection.
Related MCP server: Browser MCP Server
Tool | Description |
| Start here. Free-first fetch with automatic Bright Data fallback. Supports |
| Fetch up to 50 URLs in parallel with the same free-first routing. Returns a cost summary. Much faster than looping |
| Start at a URL and follow links up to a configurable depth and page limit. Stays on the same host by default; accepts a URL regex filter. |
| Fetch a page and compare it to the last cached version. Stores a baseline on first call; detects changes on every subsequent call. |
| Extract structured metadata from a page: JSON-LD schema.org objects, Open Graph tags, Twitter card meta, and standard meta tags. Returns clean JSON. |
| Fetch and parse an RSS 2.0 feed, Atom 1.0 feed, or XML sitemap. Returns structured JSON — no HTML parsing needed. |
| Check whether a URL is permitted by the site's |
| Dump the current skip-list of hard-blocked domains for cost debugging. |
| Web search. DuckDuckGo queries try a free direct fetch first. Google, Bing, and Yandex always use the paid Bright Data SERP API. |
Tool | Description |
| Force a fetch through Bright Data's Web Unlocker. Use when you specifically need a screenshot, or when |
| Start a long-running unlock job. Returns a |
| Poll for and retrieve the result of an async unlock job. |
| Per-domain success-rate statistics from the last 7 days. |
| Trigger a Bright Data dataset / Web Scraper crawl job over one or more URLs. |
| Check progress and download results for a crawl job by |
| Drive a real remote cloud browser: navigate, click, type, scroll, run JS, auto-solve CAPTCHAs, emulate devices, block ads, return rendered HTML / text / screenshot. Requires |
| Pre-built structured scrapers for Amazon, LinkedIn, Google Maps, Instagram, Reddit, and more. Returns clean JSON — no setup. |
git clone https://github.com/AndrewEstopinan/browser-mcp
cd browser-mcp
npm install && npm run build
node dist/index.js # works immediately — no API key needed for free toolsTo unlock paid tools, set BRIGHTDATA_API_KEY:
node --env-file=.env dist/index.jsCopy .env.example to .env and fill in what you need.
Variable | Required | Default | Description |
| No | — | Enables paid tools. Free tools work without it. |
| No |
| Web Unlocker zone name. |
| No |
| SERP API zone name. |
| For | — |
|
| No |
| CDP host. |
| For crawl tools | — | Default dataset id ( |
| No |
| Override API base. |
| No |
| Blocks before a domain is skip-listed. |
| No |
| Skip-list entry lifetime in ms (30 min). |
| No |
| Persisted skip-list path. |
| No |
| Response cache directory. |
| No |
| Default cache TTL in ms (1 hour). |
Add this to your Claude Desktop, Cursor, or other MCP client config:
{
"mcpServers": {
"scout": {
"command": "node",
"args": ["/absolute/path/to/browser-mcp/dist/index.js"],
"env": {
"BRIGHTDATA_API_KEY": "your_api_key_here",
"BRIGHTDATA_UNLOCKER_ZONE": "web_unlocker1",
"BRIGHTDATA_SERP_ZONE": "serp_api1",
"BRIGHTDATA_BROWSER_AUTH": "brd-customer-XXXX-zone-YYYY:zzzz"
}
}
}
}For free-only use, omit the env block entirely.
Fetch an article (content only, no nav/ads):
{ "tool": "smart_scrape", "arguments": { "url": "https://example.com/article", "data_format": "content_only" } }Fetch 10 URLs in parallel:
{ "tool": "smart_scrape_batch", "arguments": { "urls": ["https://a.com", "https://b.com"], "concurrency": 5 } }Crawl a docs site up to 20 pages:
{ "tool": "smart_crawl", "arguments": { "start_url": "https://docs.example.com", "max_pages": 20, "max_depth": 3 } }Check if a page changed since last time:
{ "tool": "smart_diff", "arguments": { "url": "https://example.com/pricing" } }Extract JSON-LD and Open Graph metadata:
{ "tool": "smart_extract", "arguments": { "url": "https://example.com/product/123" } }Parse an RSS feed:
{ "tool": "parse_feed", "arguments": { "url": "https://example.com/feed.xml" } }Drive a cloud browser through a CAPTCHA:
{
"tool": "browser_scrape",
"arguments": {
"url": "https://hard-site.example",
"solve_captcha": true,
"screenshot": true,
"actions": [
{ "type": "wait_for_selector", "selector": "#results" },
{ "type": "click", "selector": "button.load-more" }
]
}
}Scout accesses public web data through a paid, compliant commercial service. Respect each target site's Terms of Service and robots.txt (use the built-in check_robots tool), applicable laws, and Bright Data's Acceptable Use Policy. Don't use it to access content behind logins you aren't authorized to use, or to collect personal data without a lawful basis.
MIT
Drive a real, remote cloud browser (Puppeteer over CDP) for JavaScript-heavy sites and multi-step flows. Supports automatic CAPTCHA solving, ad blocking, device emulation, sticky sessions, clicks/typing/scrolling, and returns rendered HTML, inner text, and/or a full-page screenshot. No local browser is required - everything runs on Bright Data's infrastructure. Requires BRIGHTDATA_BROWSER_AUTH to be set.
Inputs: url (required), device, actions, block_ads, +8 more.
Fetch and parse the site's robots.txt, then check whether the given URL path is permitted for the wildcard (*) user-agent. Returns the verdict and the matching rule. robots.txt is cached for 24 hours.
Inputs: url (required).
Fetch and parse an RSS 2.0 feed, Atom 1.0 feed, or XML sitemap. Auto-detects the format from content-type and content. Returns structured JSON with items/entries or sitemap URLs. Free direct fetch — no Bright Data credit unless the feed is behind a wall.
Inputs: url (required), free_only.
Replays an HTTP Archive (.har) file exported from browser DevTools. Automatically detects dynamic values (CSRF tokens, nonces, hidden form fields, JSON token fields) in responses and substitutes fresh values into subsequent requests — so login flows, form submissions, and signed API calls work correctly even though the recorded values are stale. Static assets (images, CSS, JS) are skipped by default. Use 'substitutions' for values that never appear in responses (credentials, API keys). Use 'dry_run' to preview what would be sent.
Inputs: har_path (required), dry_run, skip_assets, substitutions.
Run a web search. For DuckDuckGo queries, tries a FREE direct fetch first and only falls back to the paid Bright Data SERP API if blocked. For Google, Bing, and Yandex, always uses the paid SERP API (no free public API exists). Returns structured JSON (parse=true) or raw HTML/Markdown (parse=false).
Inputs: query (required), gl, num, page, +7 more.
Start at a URL, follow links up to a depth/page limit, and return content from each page. Stays on the same hostname by default. Use url_filter to restrict which paths are visited.
Inputs: start_url (required), free_only, max_depth, max_pages, +5 more.
Fetch a URL and compare it to the previously cached version. Returns whether the page changed, when it was last seen, and both versions so you can inspect what's different. First call always stores a baseline; subsequent calls detect changes.
Inputs: url (required), free_only, data_format, force_bright_data.
Fetch a page and extract structured data: JSON-LD schema.org objects, Open Graph tags, Twitter card meta, standard meta description/keywords, and page title. Returns clean JSON. Useful for getting product info, article metadata, or any schema.org markup without parsing HTML.
Inputs: url (required), free_only, use_cache, force_bright_data.
PREFERRED scraping tool. Fetches a page as cheaply as possible: it first tries a FREE direct HTTP request, and only falls back to the PAID Bright Data Web Unlocker if it detects an anti-bot wall (Cloudflare, Akamai, Imperva/Incapsula, PerimeterX, DataDome, a CAPTCHA, or a 403/429/503). An in-memory skip-list remembers hard domains so repeat calls skip the doomed free attempt. Most sites cost $0. The result reports which tier was used and whether credit was spent. Use this instead of unlocker_scrape for read-only fetches; reserve browser_scrape for real JS interaction.
Inputs: url (required), method, country, headers, +12 more.
Fetch an array of URLs in parallel using the same free-first routing as smart_scrape. Returns an array of {url, success, text, tier, paid} objects. Use this instead of calling smart_scrape in a loop — it's significantly faster.
Inputs: urls (required), free_only, concurrency, data_format, +3 more.
Show the in-memory list of domains currently being sent straight to Bright Data (because the free tier was blocked recently). Useful for debugging cost.
Inputs: no named parameters.
Retrieve the result of an async unlock job. If still processing, returns a PENDING status - poll again shortly.
Inputs: response_id (required), zone.
Always spends Bright Data credit. For read-only fetches prefer smart_scrape — it tries a free direct request first and only falls back here when the site actively blocks it. Use unlocker_scrape directly only when you need screenshot output, want to force Bright Data unconditionally, or smart_scrape has already confirmed the site is hard-blocked.
Inputs: url (required), zone, method, country, +4 more.
Start an asynchronous Web Unlocker request for slow/heavy pages. Returns a response_id you later pass to unlocker_get_async_result.
Inputs: url (required), zone, country, data_format.
Return Web Unlocker success-rate statistics (past 7 days) for a domain. Use a wildcard like 'example.*' to get all TLDs.
Inputs: domain (required).
Structured Amazon product data (title, price, ratings, images, specs). Requires a product URL containing /dp/.
Inputs: url (required).
Structured Amazon review data for a product. Requires a product URL containing /dp/.
Inputs: url (required).
Structured Amazon search results for a keyword. Returns first page.
Inputs: keyword (required), domain.
Structured Apple App Store app data (rating, reviews, price, description).
Inputs: url (required).
Structured Booking.com hotel data (price, rating, amenities, location).
Inputs: url (required).
Query ChatGPT and get structured AI-generated insights about a brand or topic.
Inputs: query (required).
Structured Crunchbase company data (funding rounds, investors, founded, HQ).
Inputs: url (required).
Structured eBay listing data (price, condition, seller, bids).
Inputs: url (required).
Structured Facebook Marketplace listing data.
Inputs: url (required).
Structured Facebook post data.
Inputs: url (required).
Structured GitHub repository or file data (stars, forks, contributors, file content).
Inputs: url (required).
Structured Glassdoor company profile (rating, CEO approval, culture scores).
Inputs: url (required).
Structured Glassdoor employee reviews for a company.
Inputs: url (required).
Structured Google Play app data (rating, downloads, reviews, description).
Inputs: url (required).
Query Grok and get structured AI-generated insights about a brand or topic.
Inputs: query (required).
Structured Instagram comments for a post.
Inputs: url (required).
Structured Instagram post data (caption, likes, comments, media).
Inputs: url (required).
Structured Instagram profile data (bio, followers, following, post count).
Inputs: url (required).
Structured Instagram reel data.
Inputs: url (required).
Structured LinkedIn company profile data (industry, size, description, specialties).
Inputs: url (required).
Structured LinkedIn job listing data.
Inputs: url (required).
Structured LinkedIn person profile data (headline, experience, education, skills).
Inputs: url (required).
Structured LinkedIn post/article data.
Inputs: url (required).
Structured npm package data (latest version, README, dependencies, weekly downloads).
Inputs: package_name (required).
Query Perplexity and get structured AI-generated insights about a brand or topic.
Inputs: query (required).
Structured PyPI package data (latest version, README, dependencies, metadata).
Inputs: package_name (required).
Structured Reddit comment thread for a post.
Inputs: url (required).
Structured Reddit post data (title, body, upvotes, awards, subreddit).
Inputs: url (required).
Structured Reuters news article data (headline, body, author, published date).
Inputs: url (required).
Structured TikTok comment data for a video.
Inputs: url (required).
Structured TikTok video data (views, likes, shares, comments).
Inputs: url (required).
Structured TikTok profile data (bio, followers, following, video count).
Inputs: url (required).
Structured Walmart product data. Requires a product URL containing /ip/.
Inputs: url (required).
Structured Walmart seller profile data.
Inputs: url (required).
Structured X/Twitter post data (text, likes, retweets, replies).
Inputs: url (required).
Structured YouTube comment data for a video.
Inputs: url (required).
Structured YouTube video data (views, likes, description, tags).
Inputs: url (required).
Structured YouTube channel data (subscribers, description, videos).
Inputs: url (required).
Structured Zillow property data (price, beds, baths, sqft, listing details).
Inputs: url (required).
Structured ZoomInfo company data (employees, revenue, technologies, contacts).
Inputs: url (required).
Check progress and, when ready, download the data for a snapshot_id returned by web_scraper_trigger. If the job is still running, returns its status.
Inputs: snapshot_id (required), format.
Trigger a Bright Data dataset (Web Scraper / Crawl API) collection over one or more URLs. Returns a snapshot_id to poll with web_scraper_get_results. Requires a dataset_id (gd_...) - either passed here or via BRIGHTDATA_DATASET_ID.
Inputs: urls (required), dataset_id, extra_inputs, include_errors, +1 more.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Scored across 57 tools
Every tool has a clearly distinct target and action, from smart_scrape/unlocker_scrape (with explicit cost/fallback differences) to the large web_data_* family, where each tool is bound to a specific platform and entity. Even with 57 tools, there is no real ambiguity because descriptions precisely delimit the use case.
Naming is consistent within families: web_data_* for structured data, smart_* for cost-aware fetching, unlocker_* for paid requests, and web_scraper_* for datasets. However, the overall set mixes these prefixes with standalone names like serp_search, browser_scrape, parse_feed, check_robots, and replay_har, so no single predictable convention governs the whole server.
At 57 tools, the server far exceeds the typical well-scoped 3-15 tool range. While each endpoint offers distinct value, the sheer volume makes selection harder and suggests the server is a sprawling catalog rather than a focused toolkit, potentially overwhelming agents.
The server covers the entire scraping lifecycle: free vs paid fetch, async jobs, batching, crawling, search, structured data for dozens of major sites, diff/extract, robots checking, and HAR replay. Paired trigger/get-result tools ensure no dead ends, and the domain appears fully addressed.
Web search, browser automation, scraping, crawling and CAPTCHA solving for AI agents.
Web scraping, search, crawling, structured extraction and research for AI agents.
Cloud browser, web extraction, screenshots, and remote CDP tools for AI agents.
Live web access for agents: scrape, SERP search, crawl/map, 100+ collectors, datasets, proxies.
