DocsFeed MCP Server

Why DocsFeed MCP ?

LLMs produce stale code when APIs shift and internal libraries release updates. Pasting documentation into chat windows breaks workflow and context limits.

DocsFeed automates the pipeline:

1. Crawl & Parse: Ingests documentation from URLs/sitemaps into clean Markdown chunks.

2. Embed & Index: Indexes chunks using OpenAI, Gemini, or local Ollama embeddings with pgvector.

3. Serve via MCP: Exposes native search tools to your editor through authenticated endpoints.

Related MCP server: Modular RAG MCP Server

Features

• Automated Ingestion: Crawl sitemaps or nested documentation trees automatically.

• Native MCP Tools: AI agents query documentation using structured tools (search_docs, get_page, list_sections).

• Scoped API Keys: Each documentation feed generates an isolated endpoint with an authenticated bearer token.

• Flexible Embedding Providers: Use OpenAI, Gemini, or run entirely offline using Ollama (nomic-embed-text).

• Management UI: Built-in Next.js interface to manage crawls, verify indexed pages, and monitor sync status.

Architecture Overview

[ Doc URL / Sitemap ]
         │
         ▼
[ Crawler & Parser ] ───► [ Embeddings Engine ]
                                 │ (OpenAI / Gemini / Ollama)
                                 ▼
                     [ PostgreSQL + pgvector ]
                                 │
                                 ▼
                      [ DocsFeed MCP Server ]
                                 │
                ┌────────────────┼────────────────┐
                ▼                ▼                ▼
         Cursor / Windsurf  Claude Desktop   Custom Agents

Quickstart

Prerequisites

• Node.js >= 20.0.0

• Docker (recommended for local PostgreSQL) or a serverless PostgreSQL instance (Neon / Supabase)

1. Clone & Install Dependencies

git clone https://github.com/sajannethsara/DocsFeed-MCP.git
cd DocsFeed-MCP
npm install

2. Configure Environment

Copy the example environment file:

cp .env.example .env

Configure your database and embedding provider credentials in .env:

# Database (Local Docker instance)
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/docsfeed?schema=public"

# Embeddings (Choose one)
OPENAI_API_KEY="your-openai-key"
# GEMINI_API_KEY="your-gemini-key"
# OLLAMA_BASE_URL="http://localhost:11434"

3. Start Database & Apply Migrations

If using Docker:

npm run docker:up
npm run prisma:generate
npm run prisma:migrate
npm run prisma:seed

4. Start the Application

npm run dev

The web dashboard runs at http://localhost:3000, and the MCP backend service runs at http://localhost:4000.

Client Integration

Once an endpoint is created via the dashboard, register it inside your editor's MCP configuration file.

Configuration (mcpServers)

Add the following block to your configuration file (e.g., claude_desktop_config.json or Cursor MCP settings):

{
  "mcpServers": {
    "docsfeed-docs": {
      "url": "http://localhost:4000/mcp/<SOURCE_ID>",
      "headers": {
        "Authorization": "Bearer <YOUR_API_KEY>"
      }
    }
  }
}