Docs MCP — AI-Powered Documentation Search

An MCP (Model Context Protocol) server that searches official documentation for popular libraries, scrapes the relevant pages, and returns clean, LLM-ready text. Connect it to Claude Desktop, Cursor, or any MCP-compatible client so your AI assistant can look up up-to-date docs on demand.

What it does

The project follows a three-step pipeline:

1. Search the web — Uses the Serper API to run a Google search scoped to a library's official documentation site.

2. Fetch and clean pages — Downloads the top result URLs and strips HTML noise using Groq LLM (openai/gpt-oss-20b).

3. Return structured context — Sends back cleaned text with SOURCE: links so the calling agent can cite where the information came from.

The MCP server exposes a single tool, get_docs, that AI clients can call when they need documentation context.

Related MCP server: docs-mcp-server

Supported libraries

Prerequisites

• Python 3.12+

• uv — Python package and project manager

• Serper API key — serper.dev

• Groq API key — console.groq.com

Project structure

web-scraping/
├── mcp_server.py   # MCP server (FastMCP) — exposes the get_docs tool
├── client.py       # Example MCP client that calls get_docs and summarizes results
├── utils.py        # LLM helpers (Groq) and HTML cleaning (trafilatura)
├── test.py         # Quick sanity check for the Groq API connection
├── pyproject.toml  # Project dependencies (managed by uv)
└── .env            # API keys (create this locally — do not commit)

Setup

1. Clone and enter the project

cd /path/to/web-scraping

2. Install dependencies with uv

uv sync

This creates a .venv virtual environment and installs all packages listed in pyproject.toml.

3. Create a .env file

Create a .env file in the project root with your API keys:

SERPER_API_KEY=your_serper_api_key_here
GROQ_API_KEY=your_groq_api_key_here

Both keys are required. The server loads them via python-dotenv.

4. Verify your Groq connection (optional)

uv run test.py

You should see API Key Found: True followed by a short LLM response.

Usage

Run the MCP server directly

uv run mcp_server.py

The server communicates over stdio (standard input/output), which is how MCP clients connect to it. Running it standalone will appear to hang — that is expected; it is waiting for an MCP client.

Run the example client

From the project root:

uv run client.py

This starts the MCP server as a subprocess, calls get_docs with a sample query ("How to publish a package with uv on gitlab"), and prints a human-readable answer generated by Groq.

Connect to Claude Desktop

Add the server to your Claude Desktop MCP config.

macOS config path:

~/Library/Application Support/Claude/claude_desktop_config.json

Recommended configuration — use --directory so the server works even when Claude Desktop does not honor the cwd field:

{
  "mcpServers": {
    "docs-mcp": {
      "command": "/opt/homebrew/bin/uv",
      "args": [
        "run",
        "--directory",
        "/Users/YOUR_USERNAME/Desktop/web-scraping",
        "mcp_server.py"
      ],
      "env": {
        "SERPER_API_KEY": "your_serper_api_key_here",
        "GROQ_API_KEY": "your_groq_api_key_here"
      }
    }
  }
}

Replace /Users/YOUR_USERNAME/Desktop/web-scraping with the absolute path to this project, and update the uv path if yours differs (which uv).

Alternative — run the venv Python directly (most reliable if cwd is ignored):

{
  "mcpServers": {
    "docs-mcp": {
      "command": "/Users/YOUR_USERNAME/Desktop/web-scraping/.venv/bin/python",
      "args": [
        "/Users/YOUR_USERNAME/Desktop/web-scraping/mcp_server.py"
      ],
      "env": {
        "SERPER_API_KEY": "your_serper_api_key_here",
        "GROQ_API_KEY": "your_groq_api_key_here"
      }
    }
  }
}

After saving the config, fully quit Claude Desktop (Cmd+Q) and reopen it. The docs-mcp server should appear as connected.

MCP tool reference

get_docs

Search official documentation for a library and return cleaned text.

Parameters:

Example call (from an MCP client):

{
  "query": "How to use async with LangChain",
  "library": "langchain"
}

Returns: Cleaned documentation text with SOURCE: <url> headers for each fetched page.

How it works internally

User / AI client
      │
      ▼
  get_docs(query, library)
      │
      ├─► Serper API  ──► Google search (site-scoped to official docs)
      │
      ├─► httpx       ──► Fetch top result URLs
      │
      └─► Groq LLM    ──► Clean HTML from each page (4000-char chunks)
                │
                ▼
         Structured text + source links

• mcp_server.py — Defines the FastMCP server and the get_docs tool.

• utils.py — get_response_from_llm() calls Groq; clean_html_to_txt() uses trafilatura (available as a fallback utility).

• client.py — Demonstrates programmatic MCP usage with the official Python MCP SDK.

Troubleshooting

Failed to spawn: mcp_server.py — No such file or directory

Claude Desktop launched uv from the wrong working directory. Fix by using --directory with an absolute project path (see config above), or point directly at .venv/bin/python.

GROQ_API_KEY not found

Ensure .env exists in the project root with a valid key, or pass the key via the env block in your MCP config.

Server connects then immediately disconnects

Check Claude Desktop MCP logs for Python import errors. Run uv sync to reinstall dependencies, then test locally with uv run mcp_server.py.

Library X not supported by this tool

The library parameter must exactly match one of the supported keys: langchain, lama-index, openai, or uv.

Dependencies

Managed in pyproject.toml:

• fastmcp — MCP server framework

• mcp — MCP Python SDK

• httpx — Async HTTP client

• groq — Groq LLM API client

• python-dotenv — Load .env files

• trafilatura — HTML text extraction

License

MIT License