Skip to main content
Glama

web-docs-mcp

Local MCP server for Kilo Code / Cline / Claude Desktop — free web search + library docs fetcher with local-first caching.

Features

  • šŸ” web_search — Free DuckDuckGo HTML search (no API key required)

  • 🌐 fetch_url — Fetch any URL and convert to clean markdown (cached locally)

  • šŸ“š lib_docs — Auto-fetch README/docs from npm, PyPI, crates.io, Go, or GitHub

  • šŸ“– search_docs — Language & API documentation search biased to official docs (MDN, docs.python.org, etc.)

  • šŸ“‚ list_docs — Browse and search your local docs/ folder first (local-first approach)

Related MCP server: Basic MCP Tools

Installation

npm install
npm run build

Usage

As MCP Server

Add to your MCP client configuration (e.g., claude_desktop_config.json):

{
  "mcpServers": {
    "web-docs-mcp": {
      "command": "node",
      "args": ["/path/to/web-docs-mcp/build/index.js"],
      "env": {
        "DOCS_DIR": "/path/to/your/docs",
        "CACHE_TTL_HOURS": "168",
        "DEFAULT_SAVE_TO_DOCS": "true"
      }
    }
  }
}

Direct Execution

# Development mode
npm run dev

# Production mode
npm run start

# Build
npm run build

# Clean build artifacts
npm run clean

Configuration

All settings are configurable via environment variables:

Variable

Default

Description

DOCS_DIR

./docs

Directory where fetched markdown docs are saved

CACHE_DIR

./.cache/web-docs

Directory for TTL cache (raw HTML + fetched markdown)

CACHE_TTL_HOURS

168 (7 days)

Cache time-to-live in hours

HTTP_TIMEOUT_MS

20000

HTTP timeout per request in milliseconds

USER_AGENT

Chrome-like UA

User-Agent sent to upstream servers

DEFAULT_SAVE_TO_DOCS

true

Default behavior for saving docs to docs/ folder

DISABLE_CACHE

false

Disable caching (useful for debugging)

GITHUB_TOKEN

none

Optional GitHub token to lift rate limits

Tools

Free web search via DuckDuckGo HTML. Returns a list of {title, url, snippet}.

{
  query: string,      // Search query
  limit?: number      // Number of results (default: 8, max: 20)
}

fetch_url

Fetch a single URL and return clean markdown. Handles HTML (via turndown+GFM), JSON (pretty-printed), and plain text.

{
  url: string,                    // URL to fetch
  save?: boolean,                 // Save to docs/ folder (default: true)
  subdir?: string                 // Subdirectory under docs/ (optional)
}

lib_docs

Fetch README/docs for a library by name. Tries npm → PyPI → crates.io → Go → GitHub automatically.

{
  name: string,                   // Library name (e.g., "react", "numpy", "serde", "owner/repo")
  save?: boolean,                 // Save to docs/libraries/ (default: true)
  subdir?: string                 // Custom subdirectory (optional)
}

search_docs

Language & API documentation search. Biases results to official docs sites. LOCAL-FIRST: searches your docs/ folder first.

{
  query: string,                  // Search query
  language?: string,              // Programming language (e.g., "python", "rust", "go")
  fetch_top?: boolean,            // Fetch full markdown of top result (default: false)
  save?: boolean,                 // Save fetched content (default: true)
  subdir?: string                 // Subdirectory under docs/api/ (optional)
}

list_docs

Browse and search the local docs/ folder. Three modes:

  1. No args = list all saved docs

  2. { query } = keyword search across docs/

  3. { path } = read full body of a specific doc by relative path or slug

{
  query?: string,                 // Keyword search (optional)
  path?: string                   // Relative path to specific doc (optional)
}

Directory Structure

web-docs-mcp/
ā”œā”€ā”€ src/
│   ā”œā”€ā”€ index.ts          # Main entry point
│   ā”œā”€ā”€ config.ts         # Configuration & env vars
│   ā”œā”€ā”€ lib/              # Core utilities
│   │   ā”œā”€ā”€ anubis.ts     # Anubis PoW solver (anti-bot bypass)
│   │   ā”œā”€ā”€ cache.ts      # Local caching logic
│   │   ā”œā”€ā”€ ddg.ts        # DuckDuckGo search
│   │   ā”œā”€ā”€ fetcher.ts    # HTTP fetching
│   │   ā”œā”€ā”€ html-to-md.ts # HTML to markdown conversion
│   │   └── ...
│   └── tools/            # MCP tool implementations
│       ā”œā”€ā”€ web_search.ts
│       ā”œā”€ā”€ fetch_url.ts
│       ā”œā”€ā”€ lib_docs.ts
│       ā”œā”€ā”€ search_docs.ts
│       └── list_docs.ts
ā”œā”€ā”€ docs/                 # Saved documentation (created on demand)
│   ā”œā”€ā”€ libraries/        # Library READMEs
│   ā”œā”€ā”€ api/              # API documentation
│   ā”œā”€ā”€ guides/           # Tutorials & how-tos
│   └── ...
ā”œā”€ā”€ .cache/               # TTL cache (auto-managed)
ā”œā”€ā”€ build/                # Compiled JavaScript
└── package.json

Supported Ecosystems

  • npm — JavaScript/TypeScript packages

  • PyPI — Python packages

  • crates.io — Rust crates

  • pkg.go.dev — Go modules

  • GitHub — Any repository (owner/repo format)

Local-First Approach

This server implements a local-first strategy:

  1. All fetched content is cached with configurable TTL

  2. search_docs checks your local docs/ folder before going to the web

  3. Subsequent calls return cached results in <5ms instead of re-fetching

  4. Perfect for offline work or rate-limited environments

Requirements

  • Node.js >= 18.17

  • npm or yarn

License

MIT

Contributing

  1. Fork the repository

  2. Create a feature branch (git checkout -b feature/amazing-feature)

  3. Commit your changes (git commit -m 'Add amazing feature')

  4. Push to the branch (git push origin feature/amazing-feature)

  5. Open a Pull Request

Troubleshooting

Empty search results from DuckDuckGo

The User-Agent might be blocked. Try setting a custom one:

export USER_AGENT="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"

Rate limiting on GitHub

Add a GitHub token to lift the 60 req/hour anonymous limit:

export GITHUB_TOKEN=your_token_here

Cache issues

To disable cache temporarily:

export DISABLE_CACHE=true

Or clean the cache:

npm run clean
Install Server
F
license - not found
A
quality
B
maintenance

Maintenance

–Maintainers
–Response time
–Release cycle
–Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Servers

  • F
    license
    -
    quality
    B
    maintenance
    A self-hosted MCP server providing private web search, web page fetching, and current date/time tools, powered by a bundled SearXNG instance for API-key-free local search.
    2
  • A
    license
    -
    quality
    C
    maintenance
    A fully local MCP server that provides web search via self-hosted SearXNG and page-to-markdown conversion (static and JS-rendered), all aggregated behind a single endpoint for use with AI assistants.
    MIT
  • A
    license
    -
    quality
    B
    maintenance
    MCP server enabling local-first web search, fetch, extract, and caching with citeable excerpts, no API key required. Supports research workflows for agents and apps.
    323
    MIT

View all related MCP servers

Related MCP Connectors

  • An MCP server that gives your AI access to the source code and docs of all public github repos

  • Agent-native MCP server over the public saagarpatel.dev corpus. Read-only, stateless.

  • MCP server for accessing curated awesome list documentation

View all MCP Connectors

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/Aleksandrr/web-docs-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server