# 📚 MCP Docs Intelligence Server

<p align="center">

**An asynchronous Model Context Protocol (MCP) server for searching, extracting, cleaning, and answering questions from official AI & Python ecosystem documentation.**

<br/>

<img src="https://img.shields.io/badge/Python-3.11%2B-3776AB?style=for-the-badge&logo=python&logoColor=white" alt="Python"/>
<img src="https://img.shields.io/badge/MCP-FastMCP-6A4C93?style=for-the-badge" alt="MCP"/>
<img src="https://img.shields.io/badge/LLM-Groq-FF4F00?style=for-the-badge" alt="Groq"/>
<img src="https://img.shields.io/badge/Search-Serper-4285F4?style=for-the-badge" alt="Serper"/>
<img src="https://img.shields.io/badge/Async-httpx-005571?style=for-the-badge" alt="Async HTTP"/>
<img src="https://img.shields.io/badge/License-MIT-green?style=for-the-badge" alt="MIT"/>

</p>

<p align="center">
  <em>Ask a question. Find the official docs. Clean the content. Ground the answer.</em>
</p>

---

## ✦ Overview

**MCP Docs Intelligence Server** is a lightweight asynchronous documentation intelligence system built around the **Model Context Protocol (MCP)**.

Instead of asking an LLM to answer from potentially stale or incomplete internal knowledge, this project dynamically retrieves information from the **official documentation** of supported libraries and uses that documentation as the grounding context for the final response.

The system combines:

- **FastMCP** — MCP server and tool interface
- **Serper** — web search restricted to official documentation domains
- **Trafilatura** — extraction and cleaning of useful page content
- **Groq** — fast LLM inference
- **GPT-OSS-20B** — response generation
- **httpx** — asynchronous web requests
- **Python async/await** — non-blocking architecture
- **python-dotenv** — environment-based configuration

The result is a clean pipeline that transforms:

```text
Natural Language Query
        ↓
Official Documentation Search
        ↓
Documentation Retrieval
        ↓
HTML → Clean Text
        ↓
Context Grounding
        ↓
Groq LLM
        ↓
Human-readable Answer
```

---

# 🚀 Why This Project?

Traditional documentation workflows often look like this:

```text
Search Google
     ↓
Open multiple tabs
     ↓
Read long documentation pages
     ↓
Find the relevant section
     ↓
Understand the implementation
     ↓
Write the answer
```

This project compresses that workflow into a single intelligent interface:

```text
                    ┌─────────────────────┐
                    │   Developer Query   │
                    └──────────┬──────────┘
                               │
                               ▼
                    ┌─────────────────────┐
                    │     MCP Client      │
                    └──────────┬──────────┘
                               │
                               ▼
                    ┌─────────────────────┐
                    │   FastMCP Server    │
                    │      get_docs()     │
                    └──────────┬──────────┘
                               │
                               ▼
                    ┌─────────────────────┐
                    │   Serper Search     │
                    │ Official Docs Only  │
                    └──────────┬──────────┘
                               │
                               ▼
                    ┌─────────────────────┐
                    │   Documentation     │
                    │      Pages          │
                    └──────────┬──────────┘
                               │
                               ▼
                    ┌─────────────────────┐
                    │    Trafilatura      │
                    │ HTML → Clean Text   │
                    └──────────┬──────────┘
                               │
                               ▼
                    ┌─────────────────────┐
                    │  Grounded Context   │
                    └──────────┬──────────┘
                               │
                               ▼
                    ┌─────────────────────┐
                    │       Groq          │
                    │   GPT-OSS-20B       │
                    └──────────┬──────────┘
                               │
                               ▼
                    ┌─────────────────────┐
                    │   Final Answer +    │
                    │       Sources       │
                    └─────────────────────┘
```

---

# ✨ Key Features

| Feature | Description |
|---|---|
| 🔌 **MCP-native architecture** | Documentation retrieval is exposed as an MCP tool |
| ⚡ **Async-first** | Uses asynchronous HTTP and MCP execution |
| 🌐 **Official documentation search** | Searches within trusted documentation domains |
| 🧹 **HTML cleaning** | Converts complex web pages into usable text |
| 🧠 **Context-grounded generation** | LLM answers are constrained to retrieved documentation |
| 🔗 **Source preservation** | Source URLs are retained in the final response |
| 🤖 **Groq inference** | Fast model inference through Groq |
| 🧩 **Library-aware queries** | Supports different documentation ecosystems |
| 🛠️ **Simple extensibility** | New libraries can be added through configuration |
| 🔐 **Environment-based secrets** | API keys are loaded through `.env` |
| 💻 **CLI-friendly** | User can submit queries directly from the terminal |

---

# 🏗️ System Architecture

```mermaid
flowchart TB

    U["👨‍💻 User"]

    C["🖥️ MCP Client<br/>client.py"]

    M["🔌 FastMCP Server<br/>mcp_server.py"]

    S["🔎 Serper API"]

    D["📖 Official Documentation"]

    T["🧹 Trafilatura<br/>HTML Extraction"]

    CTX["📦 Clean Documentation Context"]

    G["⚡ Groq API<br/>GPT-OSS-20B"]

    A["💬 Grounded Answer"]

    U --> C
    C --> M
    M --> S
    S --> D
    D --> T
    T --> CTX
    CTX --> C
    C --> G
    G --> A
    A --> U
```

---

# 🔄 End-to-End Request Flow

```mermaid
sequenceDiagram

    actor User
    participant Client as MCP Client
    participant MCP as FastMCP Server
    participant Serper as Serper Search API
    participant Docs as Official Docs
    participant Extractor as Trafilatura
    participant Groq as Groq LLM

    User->>Client: Enter query + library

    Client->>MCP: call_tool("get_docs")

    MCP->>Serper: Search official documentation

    Serper-->>MCP: Search results

    loop Top documentation results
        MCP->>Docs: GET documentation page
        Docs-->>MCP: HTML content
        MCP->>Extractor: Extract readable text
        Extractor-->>MCP: Clean documentation
    end

    MCP-->>Client: Documentation context + sources

    Client->>Groq: Generate answer from context

    Groq-->>Client: Grounded response

    Client-->>User: Final answer + sources
```

---

# 🧠 Core Design Principle

The system follows a simple but important principle:

> **Retrieve first. Generate second.**

The LLM is not directly responsible for discovering documentation.

Instead:

```text
          SEARCH
             ↓
       RETRIEVE DOCS
             ↓
       CLEAN CONTENT
             ↓
      BUILD CONTEXT
             ↓
        GENERATE
```

This separation makes the system easier to reason about and reduces the possibility of unsupported answers.

---

# 🧩 Supported Libraries

The current implementation supports:

| Library | Documentation Domain |
|---|---|
| 🦜 LangChain | `python.langchain.com/docs` |
| 🦙 LlamaIndex | `docs.llamaindex.ai/en/stable` |
| 🟢 OpenAI | `platform.openai.com/docs` |
| ⚡ uv | `docs.astral.sh/uv` |

The mapping is maintained in `mcp_server.py`:

```python
docs_urls = {
    "langchain": "python.langchain.com/docs",
    "llama-index": "docs.llamaindex.ai/en/stable",
    "openai": "platform.openai.com/docs",
    "uv": "docs.astral.sh/uv",
}
```

Adding a new documentation source is intentionally simple.

Example:

```python
docs_urls = {
    "langchain": "python.langchain.com/docs",
    "llama-index": "docs.llamaindex.ai/en/stable",
    "openai": "platform.openai.com/docs",
    "uv": "docs.astral.sh/uv",
    "fastapi": "fastapi.tiangolo.com",
}
```

---

# 📁 Project Structure

```text
MCP-Docs-Intelligence/
│
├── 📄 mcp_server.py
│   └── MCP server + documentation retrieval tool
│
├── 📄 client.py
│   └── User-facing MCP client
│
├── 📄 utils.py
│   └── HTML extraction + Groq LLM utility
│
├── 📄 pyproject.toml
│   └── Project metadata and dependencies
│
├── 📄 requirements.txt
│   └── pip-compatible dependency list
│
├── 📄 README.md
│   └── Project documentation
│
├── 📄 .env
│   └── Local API credentials
│
├── 📄 .gitignore
│   └── Files excluded from Git
│
└── 📁 .venv/
    └── Python virtual environment
```

---

# 🔧 Technology Stack

## FastMCP

FastMCP provides the MCP server abstraction and exposes:

```python
@mcp.tool()
async def get_docs(query: str, library: str):
```

This makes documentation retrieval available as a reusable MCP tool.

---

## Serper

Serper is used for web discovery.

The search query is restricted to the selected documentation domain:

```python
query = f"site:{docs_urls[library]} {query}"
```

Example:

```text
site:docs.astral.sh/uv How to publish a package with uv
```

This keeps the retrieval focused on the selected documentation ecosystem.

---

## Trafilatura

Documentation pages contain navigation, scripts, menus, advertisements, and other irrelevant HTML.

Trafilatura extracts the useful text:

```python
extracted = trafilatura.extract(
    html,
    include_comments=False,
    include_tables=False,
    favor_recall=False,
)
```

Conceptually:

```text
Raw HTML
   │
   ├── Navigation
   ├── Scripts
   ├── CSS
   ├── Footer
   ├── Menus
   └── Documentation Content
                │
                ▼
         Trafilatura
                │
                ▼
       Clean Documentation
```

---

## Groq

The final answer is generated using Groq's API.

Current model:

```text
openai/gpt-oss-20b
```

The model receives:

```text
System Instructions
        +
User Query
        +
Retrieved Documentation Context
```

and produces the final human-readable answer.

---

# 🔐 Environment Variables

Create a `.env` file in the project root:

```env
GROQ_API_KEY=your_groq_api_key
SERPER_API_KEY=your_serper_api_key
```

### Required credentials

| Variable | Purpose |
|---|---|
| `GROQ_API_KEY` | Authenticates requests to Groq |
| `SERPER_API_KEY` | Authenticates web search requests |

Never commit your `.env` file.

Add this to `.gitignore`:

```gitignore
.env
.venv/
__pycache__/
*.pyc
```

---

# 📦 Installation

## 1. Clone the repository

```bash
git clone https://github.com/your-username/mcp-docs-intelligence.git
cd mcp-docs-intelligence
```

---

## 2. Create a virtual environment

### Windows

```powershell
python -m venv .venv
.venv\Scripts\activate
```

### Linux / macOS

```bash
python3 -m venv .venv
source .venv/bin/activate
```

---

## 3. Install dependencies

```bash
pip install -r requirements.txt
```

Or with `uv`:

```bash
uv sync
```

---

# 📋 Dependencies

The project requires:

```text
fastmcp>=2.12.2
groq>=0.31.1
httpx>=0.28.1
python-dotenv>=1.1.1
trafilatura>=2.0.0
mcp>=1.16.0
```

---

# ▶️ Running the Project

Start the client:

```bash
python client.py
```

The client will ask:

```text
Enter your documentation query:
```

Example:

```text
Enter your documentation query:
How do I publish a Python package with uv?

Enter library (langchain/openai/llama-index/uv):
uv
```

The client automatically launches the MCP server and executes the request.

---

# 🔎 Example Request

### Input

```text
Query:
How do I publish a package with uv on GitLab?

Library:
uv
```

### Internal processing

```text
1. Receive query
       ↓
2. Select uv documentation
       ↓
3. Build site-restricted search
       ↓
4. Query Serper
       ↓
5. Retrieve documentation pages
       ↓
6. Extract readable text
       ↓
7. Return context to client
       ↓
8. Send context to Groq
       ↓
9. Generate answer
       ↓
10. Display sources
```

---

# 🧠 Grounded Answer Strategy

The client uses a system prompt similar to:

```text
Answer ONLY using the provided context.
If information is missing, say you don't know.
Keep every 'SOURCE:' line exactly.
List sources at the end.
```

This creates a clear separation between:

```text
Retrieved Facts
      │
      ▼
Documentation Context
      │
      ▼
LLM Reasoning / Response Formatting
```

The model is instructed not to invent information that does not exist in the supplied context.

---

# 🔌 MCP Tool

The server exposes one primary tool:

```text
get_docs(query, library)
```

### Parameters

| Parameter | Type | Description |
|---|---|---|
| `query` | `string` | Documentation question |
| `library` | `string` | Supported documentation ecosystem |

### Example tool call

```json
{
  "query": "How do I create a virtual environment with uv?",
  "library": "uv"
}
```

### Example response structure

```text
SOURCE: https://docs.astral.sh/uv/...

Relevant documentation content...

SOURCE: https://docs.astral.sh/uv/...

Additional documentation content...
```

---

# 🛡️ Error Handling

The server validates the selected documentation ecosystem:

```python
if library not in docs_urls:
    raise ValueError(
        f"Library {library} not supported by this tool"
    )
```

The search request also raises HTTP errors when the external API fails:

```python
response.raise_for_status()
```

This prevents silent failures and makes external API problems visible.

---

# ⚙️ Async Architecture

The project uses Python's asynchronous execution model.

Web requests use:

```python
async with httpx.AsyncClient() as client:
```

The MCP tool is also asynchronous:

```python
async def get_docs(query: str, library: str):
```

This is important because the application is primarily I/O-bound.

Instead of blocking during network operations:

```text
Application
     │
     ├── waiting for search API
     ├── waiting for docs page
     └── waiting for another docs page
```

the async architecture allows the runtime to manage I/O efficiently.

---

# 📊 Request Lifecycle

```mermaid
flowchart LR

    Q["Query"] --> V["Validate Library"]

    V --> S["Build site:<domain> Search"]

    S --> W["Serper"]

    W --> R["Top Search Results"]

    R --> F["Fetch URLs"]

    F --> C["Clean HTML"]

    C --> X["Build Context"]

    X --> L["Groq"]

    L --> O["Grounded Answer"]
```

---

# 🧱 Component Responsibilities

| Component | Responsibility |
|---|---|
| `mcp_server.py` | Search, retrieval, cleaning orchestration and MCP tool exposure |
| `client.py` | User interaction, MCP execution and LLM call |
| `utils.py` | HTML extraction and Groq API wrapper |
| `pyproject.toml` | Package metadata and dependencies |
| `requirements.txt` | pip installation support |
| `.env` | Secrets and configuration |

---

# 🧪 Example Queries

### uv

```text
How do I create and manage a virtual environment using uv?
```

### LangChain

```text
How do I create a tool-calling agent in LangChain?
```

### LlamaIndex

```text
How do I create a vector index in LlamaIndex?
```

### OpenAI

```text
How do I use structured outputs with the OpenAI API?
```

---

# 🔄 Extending the Server

The architecture is deliberately simple to extend.

## Add another documentation source

Update:

```python
docs_urls = {
    "langchain": "python.langchain.com/docs",
    "llama-index": "docs.llamaindex.ai/en/stable",
    "openai": "platform.openai.com/docs",
    "uv": "docs.astral.sh/uv",
}
```

Example:

```python
"fastapi": "fastapi.tiangolo.com"
```

The rest of the retrieval pipeline remains unchanged.

---

# 🗺️ Future Roadmap

The current implementation establishes the foundation for a much larger documentation intelligence platform.

### Phase 1 — Current

```text
✅ MCP server
✅ Async retrieval
✅ Official documentation search
✅ HTML extraction
✅ Groq integration
✅ Source preservation
✅ Interactive client
```

### Phase 2 — Retrieval Intelligence

```text
⬜ Multi-result ranking
⬜ Duplicate URL removal
⬜ Query rewriting
⬜ Better document chunking
⬜ Relevance scoring
⬜ Retry / timeout strategy
```

### Phase 3 — Advanced RAG

```text
⬜ Embedding-based retrieval
⬜ Vector database
⬜ Semantic search
⬜ Reranking
⬜ Persistent document cache
⬜ Citation-aware generation
```

### Phase 4 — Developer Platform

```text
⬜ Web UI
⬜ IDE integration
⬜ VS Code extension
⬜ Documentation diffing
⬜ Version-aware documentation
⬜ API migration assistant
⬜ Code generation from docs
```

---

# 🔮 Possible Future Architecture

```mermaid
flowchart TB

    USER["Developer"]

    UI["Web / IDE / CLI"]

    MCP["MCP Gateway"]

    QUERY["Query Understanding"]

    SEARCH["Hybrid Retrieval"]

    WEB["Official Documentation"]

    CACHE["Document Cache"]

    VECTOR["Vector Database"]

    RERANK["Reranker"]

    CONTEXT["Context Builder"]

    LLM["Groq / LLM"]

    CITE["Citation Engine"]

    ANSWER["Final Developer Answer"]

    USER --> UI
    UI --> MCP
    MCP --> QUERY

    QUERY --> SEARCH

    SEARCH --> WEB
    SEARCH --> CACHE
    SEARCH --> VECTOR

    WEB --> CONTEXT
    CACHE --> CONTEXT
    VECTOR --> RERANK
    RERANK --> CONTEXT

    CONTEXT --> LLM
    LLM --> CITE
    CITE --> ANSWER
    ANSWER --> USER
```

---

# 🎯 Design Philosophy

This project is built around four principles:

### 1. Trust the source

Prefer authoritative documentation rather than arbitrary online content.

### 2. Retrieve before generating

The model should work from current retrieved context rather than relying entirely on memorized knowledge.

### 3. Keep components replaceable

Search, extraction, MCP, and LLM layers are separated so individual components can evolve independently.

### 4. Make the system developer-friendly

The final interface should feel like asking a senior developer who knows where the official documentation is.

---

# 🔒 Security Considerations

### API keys

Never hardcode credentials:

```python
os.getenv("GROQ_API_KEY")
os.getenv("SERPER_API_KEY")
```

### Environment files

Do not commit:

```text
.env
```

### External content

Documentation retrieved from the web should be treated as external/untrusted input. Future versions should consider:

- content size limits
- URL allowlists
- response validation
- redirect validation
- request retry policies
- rate limiting
- malicious content filtering

---

# 🧰 Troubleshooting

## `ModuleNotFoundError`

Install dependencies:

```bash
pip install -r requirements.txt
```

---

## `GROQ_API_KEY` error

Verify `.env` contains:

```env
GROQ_API_KEY=your_key_here
```

and that the project is being executed from the correct directory.

---

## `SERPER_API_KEY` error

Verify:

```env
SERPER_API_KEY=your_key_here
```

---

## MCP server appears to hang

Running:

```bash
python mcp_server.py
```

directly may appear to do nothing.

That is expected for a stdio MCP server because it waits for an MCP client to communicate with it.

Normally run:

```bash
python client.py
```

instead.

---

## No documentation results

Check:

1. The selected library is supported.
2. The Serper API key is valid.
3. The documentation domain is correct.
4. Internet connectivity is available.

---

# 📜 Environment Example

```env
# Search provider
SERPER_API_KEY=your_serper_api_key

# LLM provider
GROQ_API_KEY=your_groq_api_key
```

---

# 📌 Minimal API Flow

```text
client.py
   │
   │ call_tool()
   ▼
mcp_server.py
   │
   │ search_web()
   ▼
Serper
   │
   │ documentation URLs
   ▼
fetch_url()
   │
   │ raw HTML
   ▼
clean_html_to_txt()
   │
   │ clean text
   ▼
MCP response
   │
   ▼
Groq GPT-OSS-20B
   │
   ▼
Final answer
```

---

# 🧑‍💻 Development

For local development:

```bash
git clone https://github.com/your-username/mcp-docs-intelligence.git

cd mcp-docs-intelligence

python -m venv .venv
```

Activate the environment and install dependencies:

```bash
pip install -r requirements.txt
```

Then configure `.env` and run:

```bash
python client.py
```

---

# ✅ Project Checklist

```text
[✓] MCP server implemented
[✓] FastMCP tool exposed
[✓] Async web requests
[✓] Serper integration
[✓] Official documentation restriction
[✓] HTML extraction
[✓] Groq integration
[✓] GPT-OSS-20B support
[✓] Environment variable configuration
[✓] Interactive query input
[✓] Source preservation
[✓] GitHub documentation
[✓] Architecture diagrams
```

---

# 🌟 What Makes This Project Different?

This is not simply a web scraper.

It is a small **documentation intelligence layer** that sits between a developer and the web.

The architecture creates a clear chain of responsibility:

```text
Developer
    ↓
Question
    ↓
MCP
    ↓
Retrieval
    ↓
Official Source
    ↓
Content Extraction
    ↓
Context
    ↓
LLM
    ↓
Answer
    ↓
Source
```

That separation is the foundation for turning this prototype into a larger **developer knowledge infrastructure**.

---

# 📈 Performance Characteristics

The application is primarily I/O-bound.

Potential latency contributors include:

```text
Serper Search
     +
Documentation HTTP Requests
     +
HTML Extraction
     +
LLM Generation
```

The use of asynchronous HTTP requests reduces unnecessary blocking during network operations.

Future performance improvements can include:

```text
Caching
Parallel fetching
Connection pooling
Result deduplication
Context compression
Persistent document storage
```

---

# 🤝 Contributing

Contributions are welcome.

A typical contribution workflow:

```bash
git checkout -b feature/my-feature

# Make your changes

git add .

git commit -m "feat: add documentation source"

git push origin feature/my-feature
```

Then open a pull request.

For larger architectural changes, document:

- Problem
- Proposed solution
- Architectural impact
- Backward compatibility
- Testing strategy

---

# 📄 License

This project is licensed under the **MIT License**.

Add a `LICENSE` file containing the standard MIT License text before publishing the repository.

---

# ⭐ Acknowledgements

Built using excellent open-source and developer infrastructure technologies:

- **Model Context Protocol**
- **FastMCP**
- **Groq**
- **Serper**
- **Trafilatura**
- **httpx**
- **Python**

---

# 🧭 Final Architecture Snapshot

```text
                         ┌────────────────────┐
                         │      DEVELOPER     │
                         └─────────┬──────────┘
                                   │
                                   ▼
                         ┌────────────────────┐
                         │    MCP CLIENT      │
                         │     client.py      │
                         └─────────┬──────────┘
                                   │
                                   ▼
                    ┌────────────────────────────┐
                    │       FASTMCP SERVER       │
                    │        mcp_server.py       │
                    └─────────────┬──────────────┘
                                  │
                    ┌─────────────┴─────────────┐
                    │                           │
                    ▼                           ▼
          ┌─────────────────┐         ┌─────────────────┐
          │   SERPER API    │         │  DOCUMENTATION  │
          │  Web Discovery  │────────▶│     PAGES       │
          └─────────────────┘         └────────┬────────┘
                                               │
                                               ▼
                                     ┌────────────────────┐
                                     │    TRAFILATURA     │
                                     │   Content Cleaner  │
                                     └─────────┬──────────┘
                                               │
                                               ▼
                                     ┌────────────────────┐
                                     │  CONTEXT BUILDER   │
                                     └─────────┬──────────┘
                                               │
                                               ▼
                                     ┌────────────────────┐
                                     │       GROQ         │
                                     │    GPT-OSS-20B     │
                                     └─────────┬──────────┘
                                               │
                                               ▼
                                     ┌────────────────────┐
                                     │   GROUNDED ANSWER  │
                                     │    + SOURCES       │
                                     └────────────────────┘
```

---

<p align="center">

### ⚡ Search. Extract. Ground. Answer.

**MCP Docs Intelligence Server**

</p>