Skip to main content
Glama
saivarun161

mcp-docqa-server

by saivarun161
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Hybrid

mcp-docqa-server

CI Python 3.10+ License: MIT

An MCP server that gives any AI client semantic search over a document corpus — the retrieval half of a RAG pipeline, shipped as reusable infrastructure. Point any MCP host at it and the model can search, read, and cite your documents autonomously; generation stays in the client, retrieval lives here.

"What does the corpus say about the hour-1 sepsis bundle?"
        │
        ▼                      MCP (stdio / HTTP)
┌──────────────┐   search_documents("hour-1 sepsis bundle", k=5)   ┌───────────────┐
│  AI client   │ ────────────────────────────────────────────────► │ docqa server  │
│  via any     │ ◄──────────────────────────────────────────────── │ embed → ANN   │
│  MCP host    │     top-k chunks + titles, urls, scores           │ search → rank │
└──────────────┘                                                   └──────┬────────┘
                                                                          │
                                                          SQLite (embedded, exact)
                                                          or Postgres + pgvector (HNSW)

Why this exists

Most RAG demos hard-wire retrieval into one chatbot. Exposing retrieval through MCP inverts that: index once, query from anywhere — a desktop AI assistant, an IDE agent, a CI job, your own client. The server is deliberately boring infrastructure: typed tools, two interchangeable storage backends, pluggable embeddings, an eval harness, and loud failures where silent ones usually live (see Design decisions).

Related MCP server: brkt-gdrive-mcp

Features

  • Four typed MCP tools — search_documents, fetch_document, corpus_stats, ping — with docstrings written for the calling model, because tool descriptions are the interface.

  • Hybrid retrieval by default: semantic (vector) and keyword (BM25 / Postgres FTS) search fused with Reciprocal Rank Fusion, so exact terms embeddings blur — identifiers, drug names, error codes — still land. Callers can force vector or lexical per query.

  • Two vector stores, one contract: embedded SQLite (zero infrastructure, exact brute-force cosine + FTS5) and Postgres + pgvector (HNSW index + GIN full-text, production posture). Both pass the same behavioral test battery.

  • Pluggable embeddings: OpenAI text-embedding-3-small, or a deterministic keyless hashing embedder so a fresh clone works with no API key, no database, no network.

  • Idempotent ingestion: per-document content hashes mean re-runs skip unchanged docs — nothing gets re-embedded (or re-billed) by accident.

  • Corpus fetchers for PubMed abstracts (NCBI E-utilities), arXiv papers (Atom API), and any generic RSS 2.0 / Atom 1.0 feed (blogs, changelogs, EDGAR) — all keyless and rate-limit aware, behind one --source switch.

  • Retrieval eval harness: recall@k and MRR against a labeled testset, usable as a CI quality gate (docqa-eval --min-recall5 0.9).

  • stdio + Streamable HTTP transports, Dockerfile included.

  • CI that means it: lint, unit tests, a real MCP client round-trip over stdio, and an integration job against a live pgvector service container that ends by gating on retrieval recall.

Quickstart — 60 seconds, no API key

git clone https://github.com/saivarun161/mcp-docqa-server.git
cd mcp-docqa-server
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

docqa-ingest index --sample   # bundle of 12 healthcare docs -> SQLite index
docqa-eval                    # recall@1/3/5 + MRR against the bundled testset

You now have a working index at data/index.db. Talk to it over real MCP with the Inspector:

npx @modelcontextprotocol/inspector .venv/bin/docqa-server

Wire it into your MCP host

  1. Open your MCP host's config file — each host documents its own location (commonly under the host's application-support directory).

  2. Add the block from mcp_host_config.example.json with absolute paths — hosts launch servers from their own working directory, so relative paths break.

  3. Fully restart the host and ask: "Search the docqa corpus: what counts as stage 2 hypertension?"

The model will call search_documents, read the chunks, and answer with sources.

A real corpus

Index a few hundred abstracts on any topic from PubMed or arXiv (keyless, public data):

# medical literature (PubMed, the default source)
docqa-ingest fetch --query "semaglutide cardiovascular outcomes" --max-docs 200
# or CS/ML papers (arXiv)
docqa-ingest fetch --source arxiv --query "retrieval augmented generation" --max-docs 200
# or any RSS/Atom feed — the query is the feed URL (blogs, changelogs, EDGAR filings)
docqa-ingest fetch --source feed --query "https://export.arxiv.org/rss/cs.IR"
docqa-ingest index --corpus data/corpus.jsonl
docqa-ingest stats

Any JSONL with id, title, url, text fields works, so beyond the built-in adapters you can point it at your own notes. The generic feed adapter auto-detects RSS 2.0 vs Atom 1.0 and strips HTML from entry bodies.

Production posture

Semantic embeddings — put an OpenAI key in .env (see .env.example) and re-index; DOCQA_EMBEDDINGS=auto picks it up:

pip install -e ".[openai]"
docqa-ingest index --corpus data/corpus.jsonl --force

Postgres + pgvector — vectors move into an HNSW-indexed table; search runs inside the database:

docker compose up -d        # pgvector/pgvector:pg16 with the extension enabled
export DOCQA_STORE=pgvector DATABASE_URL=postgresql://docqa:docqa@localhost:5432/docqa
pip install -e ".[pg]"
docqa-ingest index --sample

HTTP transport — for network-reachable deployments instead of stdio:

docqa-server --transport http --host 0.0.0.0 --port 8000
# or containerized:
docker build -t mcp-docqa-server . && docker run -p 8000:8000 mcp-docqa-server

MCP tools

Tool

Arguments

Returns

search_documents

query, k=5, mode="hybrid"

top-k chunks with doc_id, title, url, text, score

fetch_document

doc_id

the full source document

corpus_stats

—

doc/chunk counts, backend, embedder that built the index

ping

—

"pong" (connectivity check)

mode selects the retrieval strategy: hybrid (default) fuses both legs, vector is semantic-only (best for paraphrased/conceptual questions), lexical is keyword-only (best when an exact term must appear).

Retrieval quality

docqa-eval retrieves for every testset question and reports where the expected document ranked:

Retrieval eval — 12 questions, k=5, mode=hybrid
store=sqlite  embedder=hash-v1-512

  [rank 1] What blood pressure reading counts as stage 2 hypertension?  (expects sample-001)
  ...
recall@1=1.00  recall@3=1.00  recall@5=1.00  MRR=1.00

Pass --mode vector|lexical|hybrid to compare retrieval strategies on the same testset. The bundled corpus is small and topically distinct, so every mode scores perfectly — that run proves the plumbing. The interesting experiments start when you index a few hundred PubMed abstracts and compare hash vs openai embeddings, or vector vs hybrid, on your own testset; CI runs the eval against a live pgvector container and fails the build if recall@5 drops below 0.9.

Design decisions

  • Hybrid retrieval fuses with RRF, not score-mixing. Vector cosine and BM25 live on incomparable scales, so blending their raw scores needs fragile per-corpus tuning. Reciprocal Rank Fusion instead combines ranks — each chunk scores Σ 1/(60 + rank) over the legs it appears in — which needs no calibration and rewards chunks both legs agree on. Each leg runs in its own engine (NumPy cosine / SQLite FTS5 / Postgres GIN); the retriever pulls a deeper candidate pool from each, then fuses.

  • Embedder identity is persisted and enforced. Vectors from different embedders live in unrelated spaces; querying an OpenAI-built index with hash vectors doesn't error mathematically — it just returns garbage. The store records which embedder built it and the retriever refuses a mismatch with an actionable message. Silent failure → loud failure.

  • Brute force is a feature at SQLite scale. Exact cosine over a few thousand chunks is milliseconds with NumPy and has zero recall loss; ANN indexes buy speed at scale, not correctness. The pgvector backend adds HNSW when the corpus outgrows brute force.

  • Chunks carry their title. Each chunk is prefixed with its document title before embedding, so a chunk ripped out of context still knows what it's about.

  • One behavioral battery, two backends. The SQLite and pgvector stores pass the identical test suite (tests/store_suite.py), which is what "interchangeable" actually means.

  • The MCP layer is tested with a real MCP client. CI spawns the server over stdio and drives it with an mcp.ClientSession — the same handshake a real MCP host performs — not by calling Python functions directly.

  • Public data only. PubMed abstracts and original sample docs. Never index proprietary or employer documents into a demo corpus.

Project structure

src/docqa/
├── server.py            # FastMCP server + tool definitions
├── retriever.py         # hybrid/vector/lexical modes + RRF fusion, embedder guard
├── embeddings.py        # OpenAIEmbedder | HashingEmbedder (keyless fallback)
├── chunking.py          # word windows with overlap
├── config.py            # env-driven settings (.env aware)
├── store/
│   ├── base.py          # VectorStore contract (vector + lexical) + embedder guard
│   ├── sqlite_store.py  # embedded: brute-force cosine + FTS5 BM25
│   └── pgvector_store.py# Postgres: pgvector HNSW + GIN full-text
├── ingest/
│   ├── pubmed.py        # NCBI E-utilities fetcher (keyless, rate-limited)
│   ├── arxiv.py         # arXiv Atom fetcher (keyless; parser is unit-tested)
│   ├── feed.py          # generic RSS 2.0 / Atom 1.0 reader (keyless; parser unit-tested)
│   ├── pipeline.py      # chunk -> embed -> upsert, content-hash idempotent
│   └── cli.py           # docqa-ingest fetch | index | stats
├── eval/run_eval.py     # recall@k + MRR, CI-gateable
└── data/                # bundled 12-doc sample corpus + labeled testset
tests/                   # unit + store battery + MCP stdio round-trip
.github/workflows/ci.yml # lint, tests, live pgvector integration + recall gate

Roadmap

  • MCP tools over stdio + Streamable HTTP

  • SQLite and pgvector backends behind one contract

  • Idempotent ingestion + PubMed fetcher

  • Eval harness with CI recall gate

  • Hybrid retrieval (BM25 + vector, reciprocal rank fusion)

  • Cross-encoder reranking stage

  • Corpus adapters: PubMed + arXiv

  • Generic RSS 2.0 / Atom 1.0 feed adapter (reads EDGAR Atom feeds, blogs, changelogs)

  • Dedicated EDGAR adapter (CIK/form filtering on top of the feed reader)

  • Bearer-token auth for the HTTP transport

License

MIT — see LICENSE.

Built by Varun Kammadanam — backend + GenAI engineer (Java, Python, AWS, RAG systems).

Available Tools

4 tools
corpus_statsA

Describe the indexed corpus: document/chunk counts, storage backend, and which embedding model built the index.

Call this first if you are unsure whether the corpus is relevant to the user's question or whether anything has been indexed at all.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description uses 'describe' implying read-only behavior, which is appropriate. No annotations exist, but the description adequately conveys non-destructive intent and output components.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, minimal and front-loaded with the key purpose, followed by a usage recommendation. No redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given zero parameters and no output schema, the description fully explains the tool's purpose and output, and provides clear usage guidance.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

No parameters exist in the schema (100% coverage), so the description doesn't need to explain them. It adds value by detailing what the tool reveals.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it describes the indexed corpus including document/chunk counts, storage backend, and embedding model. It distinguishes from siblings like fetch_document (retrieves a document) and search_documents (search), providing a specific verb and resource.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises calling this tool first when unsure about corpus relevance or whether indexing has occurred, offering clear decision context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_documentA

Fetch one full source document by its id.

Use this after search_documents to read a promising source in full instead of reasoning from a chunk. Returns id, title, url, and the complete text.

Args: doc_id: The document id exactly as returned by search_documents.

ParametersJSON Schema
NameRequiredDescriptionDefault
doc_idYes

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden. It discloses what the tool returns (id, title, url, complete text) and implies a read-only, non-destructive operation. It does not detail side effects, but none are expected for a fetch.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences with an explicit Args section. Every sentence earns its place: purpose, when to use, return content, and parameter format. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the single parameter, no output schema, and the fetch task, the description provides all necessary context: what it does, when to use it, what parameter to provide, and what is returned. Sibling tools and context signals are well-addressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description compensates by explaining doc_id as 'The document id exactly as returned by search_documents', adding critical context about provenance that the schema lacks.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the tool fetches a full source document by id. It distinguishes from search_documents which returns chunks, making the purpose unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises to use after search_documents to read a full document instead of reasoning from a chunk, and specifies that doc_id must be exactly as returned by search_documents.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

pingA

Health check. Returns 'pong' to confirm the docqa server is reachable.

Use this to verify the connection between the host and this server.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations, but description fully covers behavior: returns 'pong' on success. Sufficient for a simple tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, front-loaded with purpose. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Complete for a health check tool. Output schema likely confirms return type; description sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

No parameters; description does not need to add param info. Baseline 4 for 0-param tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states health check and return value 'pong'. Distinct from siblings that handle data operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says to use for connection verification. No need to specify when not to use; siblings are unrelated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_documentsA

Search the indexed document corpus.

Returns the k best-matching text chunks, each with its source metadata: doc_id, chunk_index, title, url, the chunk text, and a relevance score (higher is better; scores are comparable within one response, not across modes). Results may include multiple chunks from the same document. Use fetch_document with a result's doc_id to read the full source document.

Args: query: A natural-language question or search phrase. k: How many chunks to return (default 5, max 25). mode: 'hybrid' (default) fuses semantic and keyword search — best for most questions. 'vector' is semantic-only — best for paraphrased or conceptual questions. 'lexical' is keyword-only — best when the exact term must appear (identifiers, drug names, error codes).

ParametersJSON Schema
NameRequiredDescriptionDefault
kNo
modeNohybrid
queryYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Describes return structure, relevance score comparability (within one response), and multiple chunk possibility. No annotations provided, so description carries full burden; it fully discloses behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Starts with purpose, then return details, then parameter explanations. Every sentence is informative with no fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given search complexity, output schema existence, and sibling tools, description is complete. Explains result format, score limitation, and mode options.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Adds rich explanation for all three parameters: query is natural language, k has default and max, mode explains three options with use cases. Schema coverage is 0%, so description compensates completely.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states 'search the indexed document corpus' and returns best-matching chunks with metadata. Distinguishes from siblings fetch_document (full document retrieval) and corpus_stats/ping.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides context for using modes (hybrid, vector, lexical) with specific guidance on when each is best. Lacks explicit 'when not to use' but covers alternatives like fetch_document for full docs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv0.1.0
    • First observedcorpus_stats
    • First observedfetch_document
    • First observedping
    • First observedsearch_documents
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A4.6/5.0

Scored across 4 tools

Disambiguation5/5

Each tool serves a unique, clearly defined purpose: ping for health check, corpus_stats for corpus metadata, search_documents for querying, and fetch_document for retrieving full documents. No functional overlap exists.

Naming Consistency4/5

Three tools follow the verb_noun pattern (corpus_stats, fetch_document, search_documents), while 'ping' is a standalone verb. This minor deviation is acceptable as ping is a standard health check command.

Tool Count5/5

With 4 tools, the server is well-scoped for a document QA system: health check, corpus statistics, search, and document retrieval. Each tool is essential and none are extraneous.

Completeness4/5

The tool set covers the core workflow of searching and fetching documents. A minor gap is the absence of a tool to list or browse documents, but for the stated purpose of answering questions via search, it is sufficient.

Maintenance

ActivitySlowing
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • mcp-rag-server

    kwanLeeFrmVi
    A
    license
    Not graded
    quality
    D
    maintenance
    An MCP server that indexes documents and serves relevant context to LLMs via Retrieval Augmented Generation (RAG).
    25 npm
    37
    MIT

  • RAG-MCP

    alejandro-ao
    F
    license
    Not graded
    quality
    D
    maintenance
    A Retrieval Augmented Generation MCP server that ingests documents into a local vector database and enables semantic search queries.
    10
    -