Skip to main content
Glama
georgittanchev

web-tools-mcp-server

by georgittanchev
OverviewSchemaRelated ServersScoreDiscussions
TypeScript
Remote

web-tools-mcp-server

Drop-in MCP replacement for Claude's native web_search and web_fetch tools. Solves the "failed to fetch" / "failed to crawl" errors caused by IP blocks and user-agent detection on Claude's default infrastructure.

Architecture

┌──────────────────────────────────────────────────────────────────┐
│                     web-tools-mcp-server                         │
│                                                                  │
│  web_search ──────► Brave Search API ($5/1k queries)             │
│                     Your own API key = your own IP = no blocks   │
│                                                                  │
│  web_fetch  ──────► Strategy: AUTO (default)                     │
│                     │                                            │
│                     ├─ 1. Direct fetch                           │
│                     │   • Rotating real-browser User-Agents      │
│                     │   • Full browser headers (Sec-Ch-Ua, etc.) │
│                     │   • Optional proxy (Bright Data / any)     │
│                     │                                            │
│                     └─ 2. Jina Reader fallback (on 403/429/503)  │
│                        • Server-side JS rendering                │
│                        • Returns clean LLM-ready markdown        │
│                        • Free tier (no key needed)               │
└──────────────────────────────────────────────────────────────────┘

Related MCP server: ai-first-scraper-mcp

Why this beats native tools

Problem with native tools

How this MCP fixes it

Claude's IP is known and blocked by many sites

Your proxy IP (Bright Data residential) or at minimum your own server's IP

Claude's user agent is detected as a bot

Rotates through 12+ real browser UAs with matching Sec-Ch-Ua headers

JS-heavy sites return empty content

Jina Reader renders JS server-side, returns clean markdown

"Failed to fetch" with no fallback

Auto strategy tries direct → falls back to Jina automatically

Search results may be limited

Brave's 35B+ page index, you control the API key

Prerequisites

  • Node.js 18+

  • Brave Search API key — Get one here ($5 free credit/month)

  • Optional: Jina API key for higher rate limits — Get one here (free tier works without key)

  • Optional: Proxy URL (Bright Data, or any HTTP/SOCKS5 proxy)

Installation

# Clone or copy this directory
cd web-tools-mcp-server

# Install dependencies
npm install

# Build
npm run build

# Verify it compiled
ls dist/index.js

Environment Variables

Variable

Required

Description

BRAVE_API_KEY

Yes

Brave Search API subscription token

PROXY_URL

No

Proxy URL for direct fetches. Format: http://user:pass@host:port or socks5://user:pass@host:port

JINA_API_KEY

No

Jina Reader API key for higher rate limits. Free tier works without it.

HTTP_PROXY

No

Alternative to PROXY_URL (standard env var)

HTTPS_PROXY

No

Alternative to PROXY_URL (standard env var)

Bright Data proxy example

export PROXY_URL="http://brd-customer-XXXXX-zone-XXXXX:PASSWORD@brd.superproxy.io:22225"

Bright Data with residential IPs (best anti-detection)

export PROXY_URL="http://brd-customer-XXXXX-zone-residential:PASSWORD@brd.superproxy.io:22225"

Setup for Claude Code

Add to your Claude Code MCP configuration (~/.claude/claude_code_config.json or per-project .claude/config.json):

{
  "mcpServers": {
    "web-tools": {
      "command": "node",
      "args": ["/absolute/path/to/web-tools-mcp-server/dist/index.js"],
      "env": {
        "BRAVE_API_KEY": "your-brave-api-key-here",
        "PROXY_URL": "http://user:pass@proxy:port",
        "JINA_API_KEY": "optional-jina-key"
      }
    }
  }
}

Or using the CLI:

claude mcp add web-tools \
  -e BRAVE_API_KEY=your-key \
  -e PROXY_URL=http://user:pass@proxy:port \
  -- node /absolute/path/to/web-tools-mcp-server/dist/index.js

Setup for Claude AI (Desktop App)

Edit ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "web-tools": {
      "command": "node",
      "args": ["/absolute/path/to/web-tools-mcp-server/dist/index.js"],
      "env": {
        "BRAVE_API_KEY": "your-brave-api-key-here",
        "PROXY_URL": "http://user:pass@proxy:port",
        "JINA_API_KEY": "optional-jina-key"
      }
    }
  }
}

Restart the Claude desktop app after saving.

Tools

web_search

Search the web via Brave Search API.

Parameters:

  • query (string, required): Search query. 1-6 words recommended.

  • count (number, optional): Results count, 1-20. Default: 10.

  • offset (number, optional): Pagination offset. Default: 0.

  • country (string, optional): Two-letter country code (e.g., "us").

  • search_lang (string, optional): Language code (e.g., "en").

  • freshness (string, optional): Time filter — "pd", "pw", "pm", "py", or "YYYY-MM-DDtoYYYY-MM-DD".

web_fetch

Fetch a URL with anti-detection capabilities.

Parameters:

  • url (string, required): Full URL to fetch (must include https:// or http://).

  • strategy (string, optional): Fetch strategy. Default: "auto".

    • "auto": Direct fetch first, Jina Reader fallback on block.

    • "direct": Direct fetch only (UA rotation + proxy).

    • "jina": Jina Reader only (JS rendering, clean markdown output).

Fetch Strategy Decision Guide

Is the site JS-heavy or known for aggressive anti-bot?
├── Yes ──► Use strategy="jina"
└── No
    ├── Do you have a proxy configured?
    │   ├── Yes ──► Use strategy="auto" (default) — direct with proxy, Jina fallback
    │   └── No  ──► Use strategy="auto" — tries without proxy, Jina catches failures
    └── Is it a raw API/JSON endpoint?
        └── Yes ──► Use strategy="direct" — Jina would mangle the JSON

Updating User Agents

The user agent pool in src/constants.ts should be updated periodically to match current browser versions. Check whatismybrowser.com/guides/the-latest-user-agent for current strings.

Cost Estimate

Component

Cost

Notes

Brave Search API

$5/1k queries

$5 free credit/month

Jina Reader

Free tier

1M tokens free, no key needed

Bright Data (optional)

~$8-15/GB residential

Pay-as-you-go available

Typical monthly for light use

~$0-5

Under 1k searches + Jina free tier

Troubleshooting

"BRAVE_API_KEY environment variable is required" → Set the BRAVE_API_KEY env var in your MCP config.

Direct fetch always returns 403 → Configure PROXY_URL with a residential proxy (Bright Data), or rely on Jina fallback (strategy="auto").

Jina returns empty/garbage → Some sites block Jina too. Configure a Bright Data residential proxy and use strategy="direct".

"undici ProxyAgent could not be loaded" → Your Node.js version doesn't bundle undici properly. Run: npm install undici in the project directory.

License

MIT

This server cannot be installed

F
license - not found
-
quality - not tested
D
maintenance

How are these scores calculated?

Maintenance

–Maintainers
–Response time
–Release cycle
–Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/georgittanchev/mcp-search'

If you have feedback or need assistance with the MCP directory API, please join our Discord server