Skip to main content
Glama
georgittanchev

web-tools-mcp-server

by georgittanchev
OverviewSchemaRelated ServersScoreDiscussions
TypeScript
Remote

web-tools-mcp-server

Drop-in MCP replacement for Claude's native web_search and web_fetch tools. Solves the "failed to fetch" / "failed to crawl" errors caused by IP blocks and user-agent detection on Claude's default infrastructure.

Architecture

┌──────────────────────────────────────────────────────────────────┐
│                     web-tools-mcp-server                         │
│                                                                  │
│  web_search ──────► Brave Search API ($5/1k queries)             │
│                     Your own API key = your own IP = no blocks   │
│                                                                  │
│  web_fetch  ──────► Strategy: AUTO (default)                     │
│                     │                                            │
│                     ├─ 1. Direct fetch                           │
│                     │   • Rotating real-browser User-Agents      │
│                     │   • Full browser headers (Sec-Ch-Ua, etc.) │
│                     │   • Optional proxy (Bright Data / any)     │
│                     │                                            │
│                     └─ 2. Jina Reader fallback (on 403/429/503)  │
│                        • Server-side JS rendering                │
│                        • Returns clean LLM-ready markdown        │
│                        • Free tier (no key needed)               │
└──────────────────────────────────────────────────────────────────┘

Related MCP server: ai-first-scraper-mcp

Why this beats native tools

Problem with native tools

How this MCP fixes it

Claude's IP is known and blocked by many sites

Your proxy IP (Bright Data residential) or at minimum your own server's IP

Claude's user agent is detected as a bot

Rotates through 12+ real browser UAs with matching Sec-Ch-Ua headers

JS-heavy sites return empty content

Jina Reader renders JS server-side, returns clean markdown

"Failed to fetch" with no fallback

Auto strategy tries direct → falls back to Jina automatically

Search results may be limited

Brave's 35B+ page index, you control the API key

Prerequisites

  • Node.js 18+

  • Brave Search API key — Get one here ($5 free credit/month)

  • Optional: Jina API key for higher rate limits — Get one here (free tier works without key)

  • Optional: Proxy URL (Bright Data, or any HTTP/SOCKS5 proxy)

Installation

# Clone or copy this directory
cd web-tools-mcp-server

# Install dependencies
npm install

# Build
npm run build

# Verify it compiled
ls dist/index.js

Environment Variables

Variable

Required

Description

BRAVE_API_KEY

Yes

Brave Search API subscription token

PROXY_URL

No

Proxy URL for direct fetches. Format: http://user:pass@host:port or socks5://user:pass@host:port

JINA_API_KEY

No

Jina Reader API key for higher rate limits. Free tier works without it.

HTTP_PROXY

No

Alternative to PROXY_URL (standard env var)

HTTPS_PROXY

No

Alternative to PROXY_URL (standard env var)

Bright Data proxy example

export PROXY_URL="http://brd-customer-XXXXX-zone-XXXXX:PASSWORD@brd.superproxy.io:22225"

Bright Data with residential IPs (best anti-detection)

export PROXY_URL="http://brd-customer-XXXXX-zone-residential:PASSWORD@brd.superproxy.io:22225"

Setup for Claude Code

Add to your Claude Code MCP configuration (~/.claude/claude_code_config.json or per-project .claude/config.json):

{
  "mcpServers": {
    "web-tools": {
      "command": "node",
      "args": ["/absolute/path/to/web-tools-mcp-server/dist/index.js"],
      "env": {
        "BRAVE_API_KEY": "your-brave-api-key-here",
        "PROXY_URL": "http://user:pass@proxy:port",
        "JINA_API_KEY": "optional-jina-key"
      }
    }
  }
}

Or using the CLI:

claude mcp add web-tools \
  -e BRAVE_API_KEY=your-key \
  -e PROXY_URL=http://user:pass@proxy:port \
  -- node /absolute/path/to/web-tools-mcp-server/dist/index.js

Setup for Claude AI (Desktop App)

Edit ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "web-tools": {
      "command": "node",
      "args": ["/absolute/path/to/web-tools-mcp-server/dist/index.js"],
      "env": {
        "BRAVE_API_KEY": "your-brave-api-key-here",
        "PROXY_URL": "http://user:pass@proxy:port",
        "JINA_API_KEY": "optional-jina-key"
      }
    }
  }
}

Restart the Claude desktop app after saving.

Tools

web_search

Search the web via Brave Search API.

Parameters:

  • query (string, required): Search query. 1-6 words recommended.

  • count (number, optional): Results count, 1-20. Default: 10.

  • offset (number, optional): Pagination offset. Default: 0.

  • country (string, optional): Two-letter country code (e.g., "us").

  • search_lang (string, optional): Language code (e.g., "en").

  • freshness (string, optional): Time filter — "pd", "pw", "pm", "py", or "YYYY-MM-DDtoYYYY-MM-DD".

web_fetch

Fetch a URL with anti-detection capabilities.

Parameters:

  • url (string, required): Full URL to fetch (must include https:// or http://).

  • strategy (string, optional): Fetch strategy. Default: "auto".

    • "auto": Direct fetch first, Jina Reader fallback on block.

    • "direct": Direct fetch only (UA rotation + proxy).

    • "jina": Jina Reader only (JS rendering, clean markdown output).

Fetch Strategy Decision Guide

Is the site JS-heavy or known for aggressive anti-bot?
├── Yes ──► Use strategy="jina"
└── No
    ├── Do you have a proxy configured?
    │   ├── Yes ──► Use strategy="auto" (default) — direct with proxy, Jina fallback
    │   └── No  ──► Use strategy="auto" — tries without proxy, Jina catches failures
    └── Is it a raw API/JSON endpoint?
        └── Yes ──► Use strategy="direct" — Jina would mangle the JSON

Updating User Agents

The user agent pool in src/constants.ts should be updated periodically to match current browser versions. Check whatismybrowser.com/guides/the-latest-user-agent for current strings.

Cost Estimate

Component

Cost

Notes

Brave Search API

$5/1k queries

$5 free credit/month

Jina Reader

Free tier

1M tokens free, no key needed

Bright Data (optional)

~$8-15/GB residential

Pay-as-you-go available

Typical monthly for light use

~$0-5

Under 1k searches + Jina free tier

Troubleshooting

"BRAVE_API_KEY environment variable is required" → Set the BRAVE_API_KEY env var in your MCP config.

Direct fetch always returns 403 → Configure PROXY_URL with a residential proxy (Bright Data), or rely on Jina fallback (strategy="auto").

Jina returns empty/garbage → Some sites block Jina too. Configure a Bright Data residential proxy and use strategy="direct".

"undici ProxyAgent could not be loaded" → Your Node.js version doesn't bundle undici properly. Run: npm install undici in the project directory.

License

MIT

Install Server
F
license - not found
A
quality
D
maintenance

How are these scores calculated?

Maintenance

–Maintainers
–Response time
–Release cycle
–Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Tools

Related MCP Servers

  • webfetch-mcp

    simonediroma
    A
    license
    -
    quality
    C
    maintenance
    Drop-in MCP replacement for the built-in WebFetch tool. Adds domain-scoped custom HTTP headers via YAML config, with bot-block detection, HTML-to-text extraction, retries, proxies, and prompt-injection sanitization.
    1
    MIT

  • ai-first-scraper-mcp

    yubinkim444
    A
    license
    A
    quality
    B
    maintenance
    Ad-free web scraping and search exposed as 3 MCP tools. fetch_page, fetch_pages_batch, search_web. Works with Claude Desktop, Cursor, Cline.
    3
    2
    MIT

  • PowerSearch MCP

    theobjectivedad
    A
    license
    -
    quality
    C
    maintenance
    Helps AI agents search the public web and fetch content with anti-bot measures, returning clean markdown outputs suitable for citation.
    2
    Apache 2.0

  • CC Web MCP

    JcDizzy
    A
    license
    -
    quality
    B
    maintenance
    Provides web search and page fetching tools for third-party models (e.g., DeepSeek, Qwen, Kimi) in Claude Code, with configurable search backends, Markdown output, and safety boundaries.
    61
    MIT

View all related MCP servers

Related MCP Connectors

  • agentsearch-mcp

    LLM-ready web search + instant answers + URL-to-clean-text fetch for agents and RAG.

  • web-search

    Web search for AI agents — one tool across 6 engines, routed to the cheapest + cached.

  • Parallel Search MCP

    The best web search for your AI Agent

View all MCP Connectors

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/georgittanchev/mcp-search'

If you have feedback or need assistance with the MCP directory API, please join our Discord server