Skip to main content
Glama
AndrewEstopinan

Bright Data MCP Server

by AndrewEstopinan
OverviewSchemaRelated ServersScoreDiscussions
TypeScript
Remote

# Scout MCP

Free-first web access for AI agents. Bright Data only when a site actually blocks you.

Scout is a Model Context Protocol server that gives your AI agent a full browser and scraping toolkit. Most pages are fetched for free with a plain HTTP request. The paid Bright Data tier only kicks in when an anti-bot wall (Cloudflare, Akamai, CAPTCHA, etc.) is detected — which means most research workflows cost nothing.

No API key required to get started. Free tools work out of the box. Configure Bright Data only when you need it.

How routing works

Every fetch follows a cost ladder and stops at the first rung that works:

1. Free direct fetch  →  realistic browser headers, no cost
2. Bright Data        →  paid proxy + unlocker, only on detected block

An in-memory skip-list (persisted to disk) remembers which domains blocked the free tier so repeat calls go straight to Bright Data — no wasted attempt. Skip-list entries expire after 30 minutes so the server re-probes in case a site drops its protection.

Related MCP server: Thordata MCP Server

Tools

Smart (cost-aware) tools

Tool

Description

smart_scrape

Start here. Free-first fetch with automatic Bright Data fallback. Supports content_only format (main article text, strips ads/nav), disk caching, robots.txt checking, per-domain rate limiting, and PDF auto-escalation.

smart_scrape_batch

Fetch up to 50 URLs in parallel with the same free-first routing. Returns a cost summary. Much faster than looping smart_scrape.

smart_crawl

Start at a URL and follow links up to a configurable depth and page limit. Stays on the same host by default; accepts a URL regex filter.

smart_diff

Fetch a page and compare it to the last cached version. Stores a baseline on first call; detects changes on every subsequent call.

smart_extract

Extract structured metadata from a page: JSON-LD schema.org objects, Open Graph tags, Twitter card meta, and standard meta tags. Returns clean JSON.

parse_feed

Fetch and parse an RSS 2.0 feed, Atom 1.0 feed, or XML sitemap. Returns structured JSON — no HTML parsing needed.

check_robots

Check whether a URL is permitted by the site's robots.txt. Result is cached for 24 hours.

smart_scrape_skiplist

Dump the current skip-list of hard-blocked domains for cost debugging.

serp_search

Web search. DuckDuckGo queries try a free direct fetch first. Google, Bing, and Yandex always use the paid Bright Data SERP API.

Bright Data tools (always paid)

Tool

Description

unlocker_scrape

Force a fetch through Bright Data's Web Unlocker. Use when you specifically need a screenshot, or when smart_scrape has already confirmed a site is hard-blocked.

unlocker_scrape_async

Start a long-running unlock job. Returns a response_id.

unlocker_get_async_result

Poll for and retrieve the result of an async unlock job.

unlocker_success_rate

Per-domain success-rate statistics from the last 7 days.

web_scraper_trigger

Trigger a Bright Data dataset / Web Scraper crawl job over one or more URLs.

web_scraper_get_results

Check progress and download results for a crawl job by snapshot_id.

browser_scrape

Drive a real remote cloud browser: navigate, click, type, scroll, run JS, auto-solve CAPTCHAs, emulate devices, block ads, return rendered HTML / text / screenshot. Requires BRIGHTDATA_BROWSER_AUTH.

web_data_* (35 tools)

Pre-built structured scrapers for Amazon, LinkedIn, Google Maps, Instagram, Reddit, and more. Returns clean JSON — no setup.

Quickstart

git clone https://github.com/AndrewEstopinan/browser-mcp
cd browser-mcp
npm install && npm run build
node dist/index.js   # works immediately — no API key needed for free tools

To unlock paid tools, set BRIGHTDATA_API_KEY:

node --env-file=.env dist/index.js

Configuration

Copy .env.example to .env and fill in what you need.

Variable

Required

Default

Description

BRIGHTDATA_API_KEY

No

—

Enables paid tools. Free tools work without it.

BRIGHTDATA_UNLOCKER_ZONE

No

web_unlocker1

Web Unlocker zone name.

BRIGHTDATA_SERP_ZONE

No

serp_api1

SERP API zone name.

BRIGHTDATA_BROWSER_AUTH

For browser_scrape

—

USER:PASS from the Browser API zone.

BRIGHTDATA_BROWSER_HOST

No

brd.superproxy.io:9222

CDP host.

BRIGHTDATA_DATASET_ID

For crawl tools

—

Default dataset id (gd_...).

BRIGHTDATA_API_BASE_URL

No

https://api.brightdata.com

Override API base.

SMART_SKIP_THRESHOLD

No

1

Blocks before a domain is skip-listed.

SMART_SKIP_TTL_MS

No

1800000

Skip-list entry lifetime in ms (30 min).

SMART_SKIP_FILE

No

./smart_skip_list.json

Persisted skip-list path.

SMART_CACHE_DIR

No

./smart_cache

Response cache directory.

SMART_CACHE_TTL_MS

No

3600000

Default cache TTL in ms (1 hour).

Add to your MCP client

Add this to your Claude Desktop, Cursor, or other MCP client config:

{
  "mcpServers": {
    "scout": {
      "command": "node",
      "args": ["/absolute/path/to/browser-mcp/dist/index.js"],
      "env": {
        "BRIGHTDATA_API_KEY": "your_api_key_here",
        "BRIGHTDATA_UNLOCKER_ZONE": "web_unlocker1",
        "BRIGHTDATA_SERP_ZONE": "serp_api1",
        "BRIGHTDATA_BROWSER_AUTH": "brd-customer-XXXX-zone-YYYY:zzzz"
      }
    }
  }
}

For free-only use, omit the env block entirely.

Example calls

Fetch an article (content only, no nav/ads):

{ "tool": "smart_scrape", "arguments": { "url": "https://example.com/article", "data_format": "content_only" } }

Fetch 10 URLs in parallel:

{ "tool": "smart_scrape_batch", "arguments": { "urls": ["https://a.com", "https://b.com"], "concurrency": 5 } }

Crawl a docs site up to 20 pages:

{ "tool": "smart_crawl", "arguments": { "start_url": "https://docs.example.com", "max_pages": 20, "max_depth": 3 } }

Check if a page changed since last time:

{ "tool": "smart_diff", "arguments": { "url": "https://example.com/pricing" } }

Extract JSON-LD and Open Graph metadata:

{ "tool": "smart_extract", "arguments": { "url": "https://example.com/product/123" } }

Parse an RSS feed:

{ "tool": "parse_feed", "arguments": { "url": "https://example.com/feed.xml" } }

Drive a cloud browser through a CAPTCHA:

{
  "tool": "browser_scrape",
  "arguments": {
    "url": "https://hard-site.example",
    "solve_captcha": true,
    "screenshot": true,
    "actions": [
      { "type": "wait_for_selector", "selector": "#results" },
      { "type": "click", "selector": "button.load-more" }
    ]
  }
}

Responsible use

Scout accesses public web data through a paid, compliant commercial service. Respect each target site's Terms of Service and robots.txt (use the built-in check_robots tool), applicable laws, and Bright Data's Acceptable Use Policy. Don't use it to access content behind logins you aren't authorized to use, or to collect personal data without a lawful basis.

License

MIT

This server cannot be installed

F
license - not found
-
quality - not tested
B
maintenance

How are these scores calculated?

Maintenance

–Maintainers
–Response time
–Release cycle
–Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/AndrewEstopinan/browser-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server