Skip to main content
Glama
sanskritiparmendranaval

MCP PDF to PGVector

by sanskritiparmendranaval
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

Problem

Large language models have no direct access to documents, PDFs, or private knowledge bases. Building a RAG pipeline requires stitching together PDF parsing, text chunking, embedding generation, and vector database operations — a fragile, multi-step process that every agent project reinvents.

This MCP server collapses that pipeline into a single agent-callable tool: point it at a directory of PDFs, and it handles extraction, chunking, embedding (nomic-embed-text-v2-moe, 768-dim), and storage in pgvector — ready for semantic search.

Related MCP server: MyDocsMCP

Features

  • Single uv run or pip install — no project scaffolding, no boilerplate.

  • Local embeddings — nomic-embed-text-v2-moe runs on CPU via sentence-transformers, no API keys or network calls.

  • Background ingestion — index_pdfs_for_rag returns immediately; poll check_indexing_progress for updates.

  • Idempotent resume — re-running indexes only new/changed files (SHA-256 content hash).

  • Per-file error isolation — one corrupt PDF never blocks the batch.

  • Auto HNSW index — pgvector index created after ingestion for sub-10ms similarity search.

  • Production stable — uses mcp==1.28.1 (stable SDK), no pre-releases.

Architecture

┌────────────────────────────────────────────────────────────┐
│  Agent (OpenCode, Claude Desktop, etc.)                    │
│  calls index_pdfs_for_rag(pdf_directory, collection_name)  │
└─────────────────────┬──────────────────────────────────────┘
                      │ JSON-RPC (stdio)
┌─────────────────────▼──────────────────────────────────────┐
│  server.py — MCP Server (FastMCP v1)                       │
│                                                             │
│  1. Scan PDF directory (recursive glob)                     │
│  2. Compute SHA-256 hash of each file                       │
│  3. Extract text (PyMuPDF)                                  │
│  4. Recursive character chunking (1000/200 default)         │
│  5. Batch embed (nomic-embed-text-v2-moe, 768-dim)          │
│  6. Batch insert into pgvector (ON CONFLICT DO NOTHING)     │
│  7. Create HNSW index on completion                         │
└─────────────────────┬──────────────────────────────────────┘
                      │ asyncpg
┌─────────────────────▼──────────────────────────────────────┐
│  PostgreSQL + pgvector                                      │
│                                                             │
│  documents (                                                │
│    collection_name TEXT,     ← namespace for multi-tenant   │
│    file_hash TEXT,           ← SHA-256 for resume/dedup     │
│    file_path TEXT,           ← original source              │
│    chunk_index INT,          ← position within file         │
│    content TEXT,             ← chunk text                   │
│    embedding vector(768),    ← nomic embedding              │
│    metadata JSONB,           ← extensible                   │
│    UNIQUE(collection_name, file_hash, chunk_index)          │
│  )                                                          │
└────────────────────────────────────────────────────────────┘

Quick Start — Agent (OpenCode)

Add to your opencode.json or ~/.config/opencode/opencode.json:

{
  "mcp": {
    "mcp-pdf-to-pgvector": {
      "type": "local",
      "command": ["/path/to/mcp-pdf-to-pgvector/.venv/bin/python", "server.py"],
      "enabled": true
    }
  }
}

Once configured, the agent can call:

index_pdfs_for_rag(
  pdf_directory="/path/to/pdfs",
  collection_name="company-policies"
)

Then poll progress:

check_indexing_progress(task_id="...")

Quick Start — Local Testing

Prerequisites

  • Python 3.10+

  • PostgreSQL 15+ with pgvector extension

Setup

# Clone and enter
git clone <url>
cd mcp-pdf-to-pgvector

# Create virtual environment and install dependencies
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# Configure pgvector connection
cp .env.example .env
# Edit .env with your pgvector credentials

# Start the server (listens on stdio)
python server.py

Test with MCP Inspector

source .venv/bin/activate
mcp dev server.py

Opens a browser UI where you can call index_pdfs_for_rag with a PDF directory path and collection name, then monitor progress with check_indexing_progress.

Configuration

All configuration is via environment variables in .env:

Variable

Default

Description

PGVECTOR_HOST

localhost

PostgreSQL host

PGVECTOR_PORT

5432

PostgreSQL port

PGVECTOR_DATABASE

vectordb

Database name

PGVECTOR_USER

postgres

Database user

PGVECTOR_PASSWORD

""

Database password

No API keys, no model configuration. Embeddings run entirely locally.

Tools

Tool

When to use

index_pdfs_for_rag

User wants PDFs searchable via natural language (knowledge base, RAG, Q&A). Starts background ingestion, returns task_id.

check_indexing_progress

After index_pdfs_for_rag, to monitor completion or when user asks "is it done yet?"

cancel_pdf_indexing

User wants to stop a running ingestion, or a task is stuck.

Schema

CREATE TABLE documents (
    id BIGSERIAL PRIMARY KEY,
    collection_name TEXT NOT NULL,
    file_hash TEXT NOT NULL,
    file_path TEXT NOT NULL,
    chunk_index INT NOT NULL,
    content TEXT NOT NULL,
    embedding vector(768),
    metadata JSONB DEFAULT '{}'::jsonb,
    created_at TIMESTAMPTZ DEFAULT NOW(),
    UNIQUE(collection_name, file_hash, chunk_index)
);

CREATE INDEX idx_documents_collection ON documents (collection_name);

-- Auto-created after ingestion:
CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops)
    WITH (m = 16, ef_construction = 200)
    WHERE collection_name = ?;

Production Considerations

  • Embedding model (~1.9GB) is downloaded on first run to ~/.cache/huggingface/hub/. Subsequent runs use the cached copy.

  • First load takes 10–30 seconds (model download + torch import). Lifespan handler pre-loads at startup.

  • Memory: ~2GB RSS during ingestion (model + torch). Returns to ~200MB after ingestion completes.

  • PostgreSQL: Ensure max_connections is sufficient. The server uses a connection pool (min: 2, max: 8).

  • Retrieval: Add a search_indexed_content tool to query the indexed data — the schema and HNSW index are ready for it.

License

MIT

This server cannot be deployed

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Maintenance

ActivitySlowing
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • qdrant-mcp

    msstnk
    A
    license
    Not graded
    quality
    D
    maintenance
    MCP server for document ingestion and semantic search on Qdrant. Enables ingesting local documents, generating embeddings with OpenAI, and performing vector search with metadata filters.
    Apache 2.0

  • pdf-context

    mematcha
    A
    license
    A
    quality
    D
    maintenance
    A local-first MCP server that ingests PDFs, extracts structure, and provides semantic search and sequential navigation tools for AI clients to query and learn from documents.
    10
    MIT