Skip to main content
Glama
enesjakozh
PedroDnT

MCP Deep Web Research Server

by PedroDnT
OverviewSchemaRelated ServersScoreDiscussions
TypeScript
Hybrid

MCP Deep Web Research Server (v0.3.0)

Node.js Version TypeScript License: MIT

A Model Context Protocol (MCP) server for advanced web research.

Latest Changes

  • Added visit_page tool for direct webpage content extraction

  • Optimized performance to work within MCP timeout limits

    • Reduced default maxDepth and maxBranching parameters

    • Improved page loading efficiency

    • Added timeout checks throughout the process

    • Enhanced error handling for timeouts

This project is a fork of mcp-webresearch by mzxrai, enhanced with additional features for deep web research capabilities. We're grateful to the original creators for their foundational work.

Bring real-time info into Claude with intelligent search queuing, enhanced content extraction, and deep research capabilities.

Related MCP server: MCP Web Research Server

Features

  • Intelligent Search Queue System

    • Batch search operations with rate limiting

    • Queue management with progress tracking

    • Error recovery and automatic retries

    • Search result deduplication

  • Enhanced Content Extraction

    • TF-IDF based relevance scoring

    • Keyword proximity analysis

    • Content section weighting

    • Readability scoring

    • Improved HTML structure parsing

    • Structured data extraction

    • Better content cleaning and formatting

  • Core Features

    • Google search integration

    • Webpage content extraction

    • Research session tracking

    • Markdown conversion with improved formatting

Prerequisites

Installation

Installing via Smithery

To install Deep Web Research Server for Claude Desktop automatically via Smithery:

npx -y @smithery/cli install @PedroDnT/mcp-deepwebresearch --client claude

Global Installation (Recommended)

# Install globally using npm
npm install -g mcp-deepwebresearch

# Or using yarn
yarn global add mcp-deepwebresearch

# Or using pnpm
pnpm add -g mcp-deepwebresearch

Local Project Installation

# Using npm
npm install mcp-deepwebresearch

# Using yarn
yarn add mcp-deepwebresearch

# Using pnpm
pnpm add mcp-deepwebresearch

Claude Desktop Integration

After installing the package, add this entry to your claude_desktop_config.json:

Windows

{
  "mcpServers": {
    "deepwebresearch": {
      "command": "mcp-deepwebresearch",
      "args": []
    }
  }
}

Location: %APPDATA%\Claude\claude_desktop_config.json

macOS

{
  "mcpServers": {
    "deepwebresearch": {
      "command": "mcp-deepwebresearch",
      "args": []
    }
  }
}

Location: ~/Library/Application Support/Claude/claude_desktop_config.json

This config allows Claude Desktop to automatically start the web research MCP server when needed.

First-time Setup

After installation, run this command to install required browser dependencies:

npx playwright install chromium

Usage

Simply start a chat with Claude and send a prompt that would benefit from web research. If you'd like a prebuilt prompt customized for deeper web research, you can use the agentic-research prompt that we provide through this package. Access that prompt in Claude Desktop by clicking the Paperclip icon in the chat input and then selecting Choose an integration → deepwebresearch → agentic-research.

Tools

  1. deep_research

    • Performs comprehensive research with content analysis

    • Arguments:

      {
  topic: string;
  maxDepth?: number;      // default: 2
  maxBranching?: number;  // default: 3
  timeout?: number;       // default: 55000 (55 seconds)
  minRelevanceScore?: number;  // default: 0.7
}

    • Returns:

      {
  findings: {
    mainTopics: Array<{name: string, importance: number}>;
    keyInsights: Array<{text: string, confidence: number}>;
    sources: Array<{url: string, credibilityScore: number}>;
  };
  progress: {
    completedSteps: number;
    totalSteps: number;
    processedUrls: number;
  };
  timing: {
    started: string;
    completed?: string;
    duration?: number;
    operations?: {
      parallelSearch?: number;
      deduplication?: number;
      topResultsProcessing?: number;
      remainingResultsProcessing?: number;
      total?: number;
    };
  };
}

  2. parallel_search

    • Performs multiple Google searches in parallel with intelligent queuing

    • Arguments: { queries: string[], maxParallel?: number }

    • Note: maxParallel is limited to 5 to ensure reliable performance

  3. visit_page

    • Visit a webpage and extract its content

    • Arguments: { url: string }

    • Returns:

      {
  url: string;
  title: string;
  content: string;  // Markdown formatted content
}

Prompts

agentic-research

A guided research prompt that helps Claude conduct thorough web research. The prompt instructs Claude to:

  • Start with broad searches to understand the topic landscape

  • Prioritize high-quality, authoritative sources

  • Iteratively refine the research direction based on findings

  • Keep you informed and let you guide the research interactively

  • Always cite sources with URLs

Configuration Options

The server can be configured through environment variables:

  • MAX_PARALLEL_SEARCHES: Maximum number of concurrent searches (default: 5)

  • SEARCH_DELAY_MS: Delay between searches in milliseconds (default: 200)

  • MAX_RETRIES: Number of retry attempts for failed requests (default: 3)

  • TIMEOUT_MS: Request timeout in milliseconds (default: 55000)

  • LOG_LEVEL: Logging level (default: 'info')

Error Handling

Common Issues

  1. Rate Limiting

    • Symptom: "Too many requests" error

    • Solution: Increase SEARCH_DELAY_MS or decrease MAX_PARALLEL_SEARCHES

  2. Network Timeouts

    • Symptom: "Request timed out" error

    • Solution: Ensure requests complete within the 60-second MCP timeout

  3. Browser Issues

    • Symptom: "Browser failed to launch" error

    • Solution: Ensure Playwright is properly installed (npx playwright install)

Debugging

This is beta software. If you run into issues:

  1. Check Claude Desktop's MCP logs:

    # On macOS
tail -n 20 -f ~/Library/Logs/Claude/mcp*.log

# On Windows
Get-Content -Path "$env:APPDATA\Claude\logs\mcp*.log" -Tail 20 -Wait

  2. Enable debug logging:

    export LOG_LEVEL=debug

Development

Setup

# Install dependencies
pnpm install

# Build the project
pnpm build

# Watch for changes
pnpm watch

# Run in development mode
pnpm dev

Testing

# Run all tests
pnpm test

# Run tests in watch mode
pnpm test:watch

# Run tests with coverage
pnpm test:coverage

Code Quality

# Run linter
pnpm lint

# Fix linting issues
pnpm lint:fix

# Type check
pnpm type-check

Contributing

  1. Fork the repository

  2. Create your feature branch (git checkout -b feature/amazing-feature)

  3. Commit your changes (git commit -m 'Add some amazing feature')

  4. Push to the branch (git push origin feature/amazing-feature)

  5. Open a Pull Request

Coding Standards

  • Follow TypeScript best practices

  • Maintain test coverage above 80%

  • Document new features and APIs

  • Update CHANGELOG.md for significant changes

  • Follow semantic versioning

Performance Considerations

  • Use batch operations where possible

  • Implement proper error handling and retries

  • Consider memory usage with large datasets

  • Cache results when appropriate

  • Use streaming for large content

Requirements

  • Node.js >= 18

  • Playwright (automatically installed as a dependency)

Verified Platforms

  • macOS

  • Windows

  • Linux

License

MIT

Credits

This project builds upon the excellent work of mcp-webresearch by mzxrai. The original codebase provided the foundation for our enhanced features and capabilities.

Author

qpd-v

Available Tools

3 tools
deep_researchC

Perform deep research on a topic with content extraction and analysis

ParametersJSON Schema
NameRequiredDescriptionDefault
topicYesResearch topic or question
maxDepthNoMaximum depth of related content exploration
maxBranchingNoMaximum number of related paths to explore
timeoutNoResearch timeout in milliseconds
minRelevanceScoreNoMinimum relevance score for including content

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure but offers minimal information. It mentions 'deep research' with 'content extraction and analysis', hinting at a potentially resource-intensive or iterative process, but fails to detail critical aspects like execution time, rate limits, authentication needs, output format, or error handling. This leaves significant gaps for a tool with 5 parameters and no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core action ('perform deep research') and key features ('content extraction and analysis') without any wasted words. It is appropriately sized for the tool's complexity, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (5 parameters, no annotations, no output schema), the description is incomplete. It lacks details on the research methodology, output format, error conditions, or performance characteristics, which are crucial for an agent to use it effectively. The high parameter count and absence of output schema demand more contextual information than provided.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, meaning all parameters are documented in the input schema with descriptions and constraints (e.g., 'maxDepth' with min/max 1-2). The description adds no additional parameter semantics beyond implying a research process, so it meets the baseline of 3 without compensating or detracting from the schema's coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('perform deep research') and key activities ('content extraction and analysis'), which distinguishes it from sibling tools like 'parallel_search' and 'visit_page' that likely have different scopes. However, it doesn't explicitly differentiate itself from those siblings in terms of depth or methodology, keeping it from a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'parallel_search' or 'visit_page', nor does it mention prerequisites, constraints, or typical use cases. It lacks explicit when/when-not instructions or comparisons, leaving the agent to infer usage from the tool name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

visit_pageC

Visit a webpage and extract its content

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesURL to visit

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. While 'visit' and 'extract' imply read-only operations, it doesn't specify important behavioral traits like rate limits, authentication needs, timeout behavior, content format returned, error handling, or whether it follows redirects. The description is minimal and lacks operational context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise with just 7 words that directly state the tool's function. Every word earns its place, and the information is front-loaded with no unnecessary elaboration or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no annotations, no output schema, and sibling tools that likely serve related purposes, the description is insufficient. It doesn't explain what 'extract its content' means in practice, what format the content returns in, how it handles different content types, or how it differs from the research/search siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage with a clear parameter description for 'url'. The tool description doesn't add any parameter-specific information beyond what the schema already provides, so it meets the baseline score of 3 for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('visit') and resource ('webpage'), and specifies the action ('extract its content'). It distinguishes itself from potential siblings by focusing on single-page content extraction rather than research or parallel operations, though it doesn't explicitly name alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus the sibling tools 'deep_research' or 'parallel_search'. It doesn't mention any prerequisites, limitations, or contextual factors that would help an agent choose between these options.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

B3.1/5.0
Disambiguation5/5

Each tool has a clearly distinct purpose: deep_research focuses on comprehensive topic analysis, parallel_search handles multiple Google searches, and visit_page extracts content from specific webpages. There is no overlap in functionality, making tool selection straightforward for an agent.

Naming Consistency3/5

The naming is mixed: deep_research and parallel_search use snake_case with descriptive names, while visit_page also uses snake_case but is more action-oriented. There is no consistent verb_noun pattern, but the names are still readable and understandable.

Tool Count3/5

With only 3 tools, the server feels thin for a 'Deep Web Research' scope, which might imply more comprehensive capabilities like data analysis or report generation. However, the tools cover core search and extraction tasks, so it's borderline but not severely lacking.

Completeness3/5

The tools cover basic web research tasks (searching, visiting, deep analysis), but there are notable gaps such as no tools for saving results, managing research sessions, or advanced data processing. Agents can work around this, but the surface is not fully comprehensive for deep web research.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Appeared in Searches

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/PedroDnT/mcp-DEEPwebresearch'

If you have feedback or need assistance with the MCP directory API, please join our Discord server