Skip to main content
Glama

MCP WebScout

A Model Context Protocol (MCP) server providing web search (DuckDuckGo) and intelligent content extraction with LLM-powered analysis.

Features

  • search: Search the web using DuckDuckGo

  • fetch: Advanced web fetching with Crawl4AI and LLM extraction

Related MCP server: OneSearch MCP Server

System Requirements

Requirement

Version

Notes

Python

>= 3.10

Required runtime environment

pip

latest

Package manager (included with Python)

Playwright

latest

Required by Crawl4AI for browser automation

DeepSeek API Key

-

Required for LLM extraction mode

Proxy (optional)

-

Required for users in mainland China

Python Dependencies (14 packages)

Package

Version

Purpose

mcp

>=1.0.0

MCP protocol implementation

duckduckgo-search

>=3.0.0

DuckDuckGo search API

requests

>=2.32.0

HTTP requests

beautifulsoup4

>=4.12.0

HTML parsing

openai

>=1.30.0

OpenAI API client for DeepSeek

crawl4ai

>=0.5.0

Advanced web scraping

Quick Start

Get started in 5 steps:

1. Clone and Setup Environment

git clone <repository>
cd mcp-webscout
python -m venv .venv

On Windows:

.venv\Scripts\activate

On macOS/Linux:

source .venv/bin/activate

2. Install Dependencies

pip install -e ".[dev]"

3. Install Playwright Browsers

playwright install chromium

4. Configure Environment Variables

cp .env.example .env

Edit .env and add your configuration:

# Required for LLM extraction
DEEPSEEK_API_KEY=sk-your-actual-key-here

# Required for mainland China users
PROXY_URL=http://127.0.0.1:7890
USE_PROXY=true

5. Verify Installation

# Run tests
pytest tests/ -v

# Test the server
python -m mcp_webscout --help

Detailed Configuration

For detailed environment setup instructions, see ENV_SETUP.md.

Usage

As a Command

mcp-webscout

As a Python Module

python -m mcp_webscout

With Claude Desktop

Add to your claude_desktop_config.json:

Basic Configuration

{
  "mcpServers": {
    "webscout": {
      "command": "mcp-webscout"
    }
  }
}
{
  "mcpServers": {
    "webscout": {
      "command": "mcp-webscout",
      "env": {
        "DEEPSEEK_API_KEY": "sk-your-key-here",
        "PROXY_URL": "http://127.0.0.1:7890",
        "USE_PROXY": "true",
        "DEFAULT_MAX_LENGTH": "5000",
        "PYTHONUTF8": "1"
      }
    }
  }
}

Windows Configuration

{
  "mcpServers": {
    "webscout": {
      "command": "python",
      "args": ["-m", "mcp_webscout"],
      "env": {
        "DEEPSEEK_API_KEY": "sk-your-key-here",
        "PROXY_URL": "http://127.0.0.1:7890",
        "USE_PROXY": "true",
        "PYTHONUTF8": "1"
      }
    }
  }
}

Tools

Search the web using DuckDuckGo.

Parameters:

Name

Type

Required

Description

query

string

Yes

Search query

max_results

integer

No

Maximum results (1-10, default: 5)

Returns:

Formatted search results with titles, URLs, and snippets.

Example:

{
  "query": "Python programming",
  "max_results": 3
}

fetch

Advanced web fetching with Crawl4AI and LLM extraction.

Parameters:

Name

Type

Required

Description

url

string

Yes

URL to fetch

mode

string

No

Extraction mode: simple, llm (default: simple)

prompt

string

No

Custom extraction prompt for LLM mode

max_length

integer

No

Maximum characters (default: 5000)

use_proxy

boolean

No

Use proxy (default: true)

Returns:

Fetched and optionally extracted content.

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    A
    maintenance
    A Model Context Protocol (MCP) server that provides web search capabilities through DuckDuckGo, with additional features for content fetching and parsing.
    3
    15,214 PyPI
    1,510
    MIT
  • A
    license
    B
    quality
    C
    maintenance
    A Model Context Protocol server that enables AI assistants to perform real-time web searches, retrieving up-to-date information from the internet via a Crawler API.
    1
    97 npm
    41
    ISC