Skip to main content
Glama
Nafi-Jza

WebSearch MCP Server

by Nafi-Jza
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Local

WebSearch MCP Server

A powerful Model Context Protocol (MCP) server that provides full-featured web searching, scraping, and YouTube transcript extraction capabilities using Playwright.

This server allows AI assistants (like Claude via Claude Code) to autonomously search the web, read articles, and fetch video transcripts without being blocked by CAPTCHAs or regional firewalls.

Features

  • 🔍 DuckDuckGo Search: Headed search using Playwright to bypass aggressive bot-protection and regional blocks (e.g., Internet Positif).

  • 🕸️ Headless Scraping: Extracts clean Markdown from websites using Mozilla's Readability and Turndown.

  • 🎥 YouTube Transcripts: Fetches auto-generated or manual transcripts with a fallback to Playwright extraction if the API fails.

  • 🔓 Manual Browser Interaction: Allows the AI to open a headed browser window, pausing execution so you can manually solve CAPTCHAs or log into accounts.

  • 🍪 Persistent Profiles: Maintains a browser-profile to save your cookies, logins, and session state across runs.

Related MCP server: Web Search MCP

Prerequisites

Installation

  1. Clone the repository:

    git clone https://github.com/nafi-jza/websearch-mcp.git
cd websearch-mcp

  2. Install dependencies:

    npm install

  3. Install Playwright browsers:

    npx playwright install chromium

  4. Build the project:

    npm run build

Configuration in Claude Code

To add this MCP server to your Claude Code setup, edit your ~/.claude.json or project-level .claude.json file:

{
  "mcpServers": {
    "websearch-mcp": {
      "type": "stdio",
      "command": "node",
      "args": [
        "C:/absolute/path/to/websearch-mcp/dist/index.js"
      ],
      "env": {}
    }
  }
}

(Make sure to replace the path with your actual absolute path to the dist/index.js file)

Available Tools

Once configured, the following tools will be available to the AI:

  • search: Search the web using DuckDuckGo.

    • Input: { query: string }

  • scrape: Load a webpage and extract its main content as Markdown.

    • Input: { url: string }

  • open_browser: Open a headed browser window to manually solve CAPTCHAs or log into sites. Pauses the MCP server until you close the browser.

    • Input: { url?: string }

  • youtube_transcript: Extract the transcript from a YouTube video.

    • Input: { target: string, lang?: string }

Architecture & How It Works

  • Bot Bypass: Playwright is used to execute searches visibly (headed) which tricks most basic bot-protection systems and DNS blocks.

  • Graceful Cleanup: The server automatically manages Chromium contexts. It safely removes the default about:blank pages and closes browser instances strictly after automated executions to prevent memory leaks and zombie processes.

  • Logging & Output: All activity is logged to activity.log in the root directory. Markdown outputs of scrapes are saved locally to the outputs/ directory for historical reference.

Available Tools

4 tools
open_browserA

Open a headed browser window to manually solve CAPTCHAs or log into sites. Pauses the MCP server until the user closes the browser.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlNoAn optional URL to navigate to when the browser opens.

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses blocking nature (pauses server) and purpose, but lacks details on prerequisites (e.g., GUI environment), failure modes, or security implications. No annotations to supplement.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with primary action and purpose. Every word earns its place; no repetition or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Adequate for a simple tool with one optional param and no output schema. Could mention that it requires a graphical display environment, but overall complete enough for a competent agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with exact same description as in schema. Tool description adds no extra meaning beyond what the schema provides for the single parameter, so baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states it opens a headed browser for manual CAPTCHAs or login, distinguishing it from sibling tools like scrape (automated). Verb+resource+use case explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies use when human intervention for CAPTCHAs or login is needed. Context is clear, but no explicit exclusions or alternative comparisons to siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

scrapeA

Load a webpage and extract its main content as Markdown.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL of the webpage to scrape.

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Without annotations, the description bears full responsibility for behavioral disclosure. It fails to mention limitations such as handling of dynamic content, authentication, rate limits, or what 'main content' means. This lack of detail hampers informed selection.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence conveys the tool's action and output efficiently. Every word serves a purpose, with no redundancy or extraneous information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description is adequate but lacks details on return format, error handling, or limitations. It provides minimal completeness beyond the obvious.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the schema already documents the 'url' parameter thoroughly. The description doesn't add meaning beyond the schema's own description, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Load and extract' with a clear resource 'webpage' and output format 'Markdown'. It effectively distinguishes from siblings like open_browser (for interactive browsing) and search (for searching), as scraping is uniquely for extracting content as Markdown.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for converting webpages to Markdown but offers no explicit guidance on when to use this tool versus alternatives (e.g., open_browser for interactive access, search for searching). No when-not-to-use or prerequisites are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

youtube_transcriptC

Extract the transcript from a YouTube video.

ParametersJSON Schema
NameRequiredDescriptionDefault
langNoOptional preferred language code (e.g., 'en', 'es').
targetYesThe YouTube video URL or ID.

TDQS

C2.8/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description fails to disclose any behavioral traits such as authentication requirements, availability of transcripts for all videos, or response format. This is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, which is concise but under-specified. It lacks important details, so it does not fully earn its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema and no annotations, the description should include more context about what the tool returns, potential errors, or usage limits. It is incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the description does not add new meaning beyond what the schema already provides for the two parameters. Baseline score is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Extract the transcript from a YouTube video' uses a specific verb and resource, clearly distinguishing it from sibling tools like open_browser, scrape, and search.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives like scrape or search. The description does not mention limitations or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv1.0.0
    • First observedopen_browser
    • First observedscrape
    • First observedsearch
    • First observedyoutube_transcript
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A3.5/5.0

Scored across 4 tools

Disambiguation5/5

Each tool serves a clearly distinct purpose: open_browser for manual interaction, scrape for content extraction, search for web search, and youtube_transcript for YouTube transcripts. No overlap.

Naming Consistency3/5

Naming is inconsistent: scrape and search are single verbs, open_browser follows verb_noun, while youtube_transcript uses noun_noun. This lack of a uniform pattern could confuse an agent.

Tool Count5/5

With 4 tools, the server is well-scoped. Each tool provides essential functionality for web research without unnecessary bloat.

Completeness4/5

The tool set covers core web search, scraping, and transcript extraction, but lacks features like image search or search history. Still, it handles common workflows effectively.

Maintenance

ActivityStale
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • MCP MyZ Search

    kimmywork
    A
    license
    Not graded
    quality
    F
    maintenance
    Enables web search across Bing, DuckDuckGo, and Google using Playwright automation, with tools for retrieving search results and extracting page text content.
    6 npm
    MIT

  • Web Search MCP

    pranavms13
    A
    license
    Not graded
    quality
    D
    maintenance
    Enables web searching through Google, DuckDuckGo, and Bing using a headless Chrome browser, returning structured results with titles, URLs, and snippets. Also supports fetching and extracting text content from any webpage.
    14
    MIT

  • Web Research MCP Server

    CSOAI-ORG
    A
    license
    C
    quality
    B
    maintenance
    Enables AI agents to perform web searches via DuckDuckGo without API keys and automate browser tasks with headless Chromium. Supports multi-step research workflows that automatically search and extract content from multiple pages.
    7
    38 npm
    26 PyPI
    MIT