web-docs-mcp
Provides free web search via DuckDuckGo, returning titles, URLs, and snippets.
Fetches README and documentation files from GitHub repositories.
Automatically fetches README and documentation for npm packages.
Automatically fetches README and documentation for PyPI packages.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@web-docs-mcpFetch the README for the 'requests' Python library"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
web-docs-mcp
Local MCP server for Kilo Code / Cline / Claude Desktop ā free web search + library docs fetcher with local-first caching.
Features
š web_search ā Free DuckDuckGo HTML search (no API key required)
š fetch_url ā Fetch any URL and convert to clean markdown (cached locally)
š lib_docs ā Auto-fetch README/docs from npm, PyPI, crates.io, Go, or GitHub
š search_docs ā Language & API documentation search biased to official docs (MDN, docs.python.org, etc.)
š list_docs ā Browse and search your local
docs/folder first (local-first approach)
Related MCP server: Basic MCP Tools
Installation
npm install
npm run buildUsage
As MCP Server
Add to your MCP client configuration (e.g., claude_desktop_config.json):
{
"mcpServers": {
"web-docs-mcp": {
"command": "node",
"args": ["/path/to/web-docs-mcp/build/index.js"],
"env": {
"DOCS_DIR": "/path/to/your/docs",
"CACHE_TTL_HOURS": "168",
"DEFAULT_SAVE_TO_DOCS": "true"
}
}
}
}Direct Execution
# Development mode
npm run dev
# Production mode
npm run start
# Build
npm run build
# Clean build artifacts
npm run cleanConfiguration
All settings are configurable via environment variables:
Variable | Default | Description |
|
| Directory where fetched markdown docs are saved |
|
| Directory for TTL cache (raw HTML + fetched markdown) |
|
| Cache time-to-live in hours |
|
| HTTP timeout per request in milliseconds |
| Chrome-like UA | User-Agent sent to upstream servers |
|
| Default behavior for saving docs to |
|
| Disable caching (useful for debugging) |
| none | Optional GitHub token to lift rate limits |
Tools
web_search
Free web search via DuckDuckGo HTML. Returns a list of {title, url, snippet}.
{
query: string, // Search query
limit?: number // Number of results (default: 8, max: 20)
}fetch_url
Fetch a single URL and return clean markdown. Handles HTML (via turndown+GFM), JSON (pretty-printed), and plain text.
{
url: string, // URL to fetch
save?: boolean, // Save to docs/ folder (default: true)
subdir?: string // Subdirectory under docs/ (optional)
}lib_docs
Fetch README/docs for a library by name. Tries npm ā PyPI ā crates.io ā Go ā GitHub automatically.
{
name: string, // Library name (e.g., "react", "numpy", "serde", "owner/repo")
save?: boolean, // Save to docs/libraries/ (default: true)
subdir?: string // Custom subdirectory (optional)
}search_docs
Language & API documentation search. Biases results to official docs sites. LOCAL-FIRST: searches your docs/ folder first.
{
query: string, // Search query
language?: string, // Programming language (e.g., "python", "rust", "go")
fetch_top?: boolean, // Fetch full markdown of top result (default: false)
save?: boolean, // Save fetched content (default: true)
subdir?: string // Subdirectory under docs/api/ (optional)
}list_docs
Browse and search the local docs/ folder. Three modes:
No args = list all saved docs
{ query }= keyword search across docs/{ path }= read full body of a specific doc by relative path or slug
{
query?: string, // Keyword search (optional)
path?: string // Relative path to specific doc (optional)
}Directory Structure
web-docs-mcp/
āāā src/
ā āāā index.ts # Main entry point
ā āāā config.ts # Configuration & env vars
ā āāā lib/ # Core utilities
ā ā āāā anubis.ts # Anubis PoW solver (anti-bot bypass)
ā ā āāā cache.ts # Local caching logic
ā ā āāā ddg.ts # DuckDuckGo search
ā ā āāā fetcher.ts # HTTP fetching
ā ā āāā html-to-md.ts # HTML to markdown conversion
ā ā āāā ...
ā āāā tools/ # MCP tool implementations
ā āāā web_search.ts
ā āāā fetch_url.ts
ā āāā lib_docs.ts
ā āāā search_docs.ts
ā āāā list_docs.ts
āāā docs/ # Saved documentation (created on demand)
ā āāā libraries/ # Library READMEs
ā āāā api/ # API documentation
ā āāā guides/ # Tutorials & how-tos
ā āāā ...
āāā .cache/ # TTL cache (auto-managed)
āāā build/ # Compiled JavaScript
āāā package.jsonSupported Ecosystems
npm ā JavaScript/TypeScript packages
PyPI ā Python packages
crates.io ā Rust crates
pkg.go.dev ā Go modules
GitHub ā Any repository (owner/repo format)
Local-First Approach
This server implements a local-first strategy:
All fetched content is cached with configurable TTL
search_docschecks your localdocs/folder before going to the webSubsequent calls return cached results in <5ms instead of re-fetching
Perfect for offline work or rate-limited environments
Requirements
Node.js >= 18.17
npm or yarn
License
MIT
Contributing
Fork the repository
Create a feature branch (
git checkout -b feature/amazing-feature)Commit your changes (
git commit -m 'Add amazing feature')Push to the branch (
git push origin feature/amazing-feature)Open a Pull Request
Troubleshooting
Empty search results from DuckDuckGo
The User-Agent might be blocked. Try setting a custom one:
export USER_AGENT="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"Rate limiting on GitHub
Add a GitHub token to lift the 60 req/hour anonymous limit:
export GITHUB_TOKEN=your_token_hereCache issues
To disable cache temporarily:
export DISABLE_CACHE=trueOr clean the cache:
npm run cleanMaintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Servers
- AlicenseAqualityAmaintenanceA local-first, no-API-key MCP server that enables LLMs to search the web, fetch pages, and read documents using multiple engines and smart fallbacks.1048MIT
- Flicense-qualityBmaintenanceA self-hosted MCP server providing private web search, web page fetching, and current date/time tools, powered by a bundled SearXNG instance for API-key-free local search.2
- Alicense-qualityCmaintenanceA fully local MCP server that provides web search via self-hosted SearXNG and page-to-markdown conversion (static and JS-rendered), all aggregated behind a single endpoint for use with AI assistants.MIT
- Alicense-qualityBmaintenanceMCP server enabling local-first web search, fetch, extract, and caching with citeable excerpts, no API key required. Supports research workflows for agents and apps.323MIT
Related MCP Connectors
An MCP server that gives your AI access to the source code and docs of all public github repos
Agent-native MCP server over the public saagarpatel.dev corpus. Read-only, stateless.
MCP server for accessing curated awesome list documentation
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/Aleksandrr/web-docs-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server