Local RAG
This server lets you build and query a private, offline vector index of your local documents without any embedding API.
Sync and ingest documents: Index PDF, DOCX, TXT, and Markdown files from configured roots via
sync_start(withsync_statusprogress polling), or add single files withingest_file; ingest in-memory text, Markdown, or client-fetched HTML withingest_datausing a stable source identifier.Hybrid search: Run
query_documentsfor keyword + semantic matching with a limit of 1–100 results, optional absolute pathscopeprefixes, and returned text, file path, chunk index, score, and heading/line/page context.Read around results: Use
read_chunk_neighborsto pull the chunks before and after a hit (up to 50 each) for surrounding context.Manage the index: List supported on-disk files and ingested
ingest_datasources withlist_files, delete indexed files or sources withdelete_file, and check index health (documentCount,chunkCount, memory, uptime, search mode) withstatus.Optional extras: Generate VLM captions for PDF figures (
visual/visualQuality) and store detected images when enabled, all within the security boundary of the configured document roots.
Provides specialized support for ingesting and indexing Markdown documents, preserving the integrity of code blocks and structural elements for improved semantic search and retrieval.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Local RAG find the error handling section in our API docs"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
MCP Local RAG
Search private documents from an MCP client or the terminal without sending them to an embedding API.
mcp-local-rag indexes PDF, DOCX, Markdown, and text files on your machine. Search combines semantic similarity with keyword matching, so queries can match both intent and exact technical terms such as API names, class names, and error codes. Results include source passages and, where available, headings, line numbers, or page numbers so you can check and cite the original document.
No API key, Docker, Python, or external database is required. After the initial model download, text ingestion and search work offline.
Quick Start
Requirements
Node.js 22.20 or later
Internet access on first use to download the npm package and embedding model
A directory containing the documents you want to search
Set BASE_DIR to that directory. It is also the security boundary for file operations. Replace
/absolute/path/to/your/documents below with the directory's absolute path.
Use one of the examples below, or register npx -y mcp-local-rag and set BASE_DIR using your
client's MCP configuration format.
Set DB_PATH and CACHE_DIR to absolute paths as well. Relative paths resolve from the
server's working directory, so starting the server from different projects creates a separate
index and model cache in each.
Run this command:
claude mcp add local-rag --scope user --env BASE_DIR=/absolute/path/to/your/documents -- npx -y mcp-local-ragAdd to ~/.codex/config.toml:
[mcp_servers.local-rag]
command = "npx"
args = ["-y", "mcp-local-rag"]
[mcp_servers.local-rag.env]
BASE_DIR = "/absolute/path/to/your/documents"Add to ~/.config/opencode/opencode.json (or opencode.jsonc):
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"local-rag": {
"type": "local",
"command": ["npx", "-y", "mcp-local-rag"],
"environment": {
"BASE_DIR": "/absolute/path/to/your/documents"
}
}
}
}Add to ~/.cursor/mcp.json:
{
"mcpServers": {
"local-rag": {
"command": "npx",
"args": ["-y", "mcp-local-rag"],
"env": {
"BASE_DIR": "/absolute/path/to/your/documents"
}
}
}
}Restart the client, then ask it to build the index:
Sync all documents in the configured root and wait until it finishes.The first sync downloads the default embedding model (about 90 MB) and may take 1–2 minutes before ingestion starts. Later runs use the local cache.
Once the sync completes:
What does the API documentation say about authentication?CLI Quick Start
To use the CLI without an MCP client:
npx mcp-local-rag ingest ./docs/
npx mcp-local-rag query "authentication API"The CLI uses the current directory as its document root by default. Run both commands from the
same directory so they use the same default index, or set BASE_DIR and DB_PATH explicitly.
Related MCP server: rag-mcp
Supported Content
Input | How to ingest |
PDF, DOCX, TXT, Markdown | File ingestion or directory sync |
HTML already fetched by the client |
|
Plain text or Markdown held in memory |
|
HTML fetching is not built into the server. An MCP client can fetch a page and pass its HTML to
ingest_data.
Excel, PowerPoint, standalone images, and source-code file extensions are not supported by file ingestion. PDFs can optionally use a local vision model to describe figures, but this is not OCR or image search.
Using the Index
Sync after adding, editing, or removing documents. For searches and follow-up reading, ask your MCP client:
Find the documented behavior of ERR_CONNECTION_REFUSED.
Read the surrounding chunks for that result.You can also ingest a single file or HTML already fetched by the client. To refresh an existing
entry, ingest the document again using the same path or source identifier. sync skips unchanged
files. MCP file paths must be absolute and inside a configured document root.
PDF section headings may be inaccurate. Check the original page when you need the exact heading.
Tool | Purpose |
| Reconcile the index with all configured roots or one path |
| Poll a running sync job |
| Ingest or replace one file |
| Ingest text, Markdown, or HTML already held by the client |
| Search with semantic matching and keyword boost |
| Read surrounding chunks from a search result |
| Show supported files and their ingestion state |
| Delete an indexed file or an |
| Show index and search status |
CLI
Use the CLI to update the index, narrow searches, or remove indexed content:
npx mcp-local-rag sync ./docs/
npx mcp-local-rag query "auth" --scope /docs/api --scope /docs/guide
npx mcp-local-rag read-neighbors --file-path /abs/path.md --chunk-index 5
npx mcp-local-rag list
npx mcp-local-rag status
npx mcp-local-rag delete ./docs/old.pdf
npx mcp-local-rag delete --source "https://example.com/docs"ingest imports the selected files; sync also removes entries for deleted files and skips
unchanged files. Use --scope to restrict search results to a path prefix, repeating it to
include multiple prefixes.
Global options such as --db-path, --cache-dir, and --model-name go before the subcommand.
Subcommand options go after it:
npx mcp-local-rag --db-path ./my-db query "authentication"Run npx mcp-local-rag --help for the complete command reference.
query writes its results to stdout as JSON, best match first, so it can be piped into another
tool. The field-by-field contract is in
docs/schema/query-output.schema.json.
If you move a project and want to keep its index, stop the MCP server and other database writers, move the files and database together without changing the directory layout, then run:
npx mcp-local-rag --db-path /new/project/lancedb relocate --from /old/project --to /new/projectUpdate BASE_DIR/BASE_DIRS, DB_PATH, and any other moved paths in your MCP configuration,
then restart the client.
Agent Skills
Agent Skills provide query and ingestion guidance for AI assistants:
npx mcp-local-rag skills install --claude-code
npx mcp-local-rag skills install --claude-code --global
npx mcp-local-rag skills install --codexInstalled skills cover query formulation, result refinement, and HTML ingestion. Ask the assistant to use the mcp-local-rag skill explicitly if it does not activate automatically.
Advanced Options
Start with the defaults. Open the sections below when you need different document roots, better results for your corpus, or searchable PDF figures.
The MCP server reads environment variables. The CLI accepts the listed variables and flags.
Keep the same DB_PATH when commands should use the same index.
Environment Variable | CLI Flag | Default | Description |
|
| Current directory | One document root; the CLI flag is repeatable on |
| N/A | (unset) | JSON array of document roots; takes precedence over |
| N/A | (unset) | JSON array of glob patterns for files to keep out of the index |
|
|
| Vector database location |
|
|
| Model cache directory |
| N/A |
| Hugging Face model download endpoint; use a mirror URL when direct downloads are blocked |
|
|
| Maximum file size in bytes |
File operations stay within configured roots. For multiple directories, set
BASE_DIRS='["/absolute/docs","/absolute/specs"]' or repeat CLI --base-dir. Precedence: CLI
roots, BASE_DIRS, BASE_DIR, then the current directory. Only the highest-priority source is
used; roots from different sources are not merged. Invalid BASE_DIRS is an error. Relative
DB_PATH and CACHE_DIR are resolved from the working directory.
To keep files out of the index, set IGNORE_PATHS to a JSON array of glob patterns, for example
IGNORE_PATHS='["plugs/**","**/*.draft.md"]'. Each pattern is matched against the file's path
relative to the root that contains it, written with / on every OS. ingest, list, and sync
skip matching files when they scan a directory, and sync removes any that are already indexed.
Invalid IGNORE_PATHS is an error.
Choose an embedding model for your documents’ language and subject. Compare settings using questions you actually ask and check which source passages are returned. The model must support mean pooling and L2 normalization, which this tool uses to produce embeddings.
Environment Variable | CLI Flag | Default | Description |
|
|
| Hugging Face embedding model |
|
|
| Minimum length in characters (1–10000) for ordinary chunks; a fragment of content split to fit the model's token limit can be shorter |
| N/A |
| Add the document title to each chunk's embedding input |
| N/A |
| Add the heading hierarchy to each chunk's embedding input when it fits |
| N/A |
| ONNX Runtime execution device |
| N/A |
| Embedding dtype passed to the selected model |
Both prefix options default to false and work independently. Try EMBED_TITLE_PREFIX when a
passage needs the document’s overall topic, or EMBED_HEADING_PREFIX when it needs its
section’s topic. Enabling both is not always better. They affect embeddings, not the returned
text or keyword index; heading context is omitted when it would exceed the input budget.
When changing embedding models, build a fresh index at a new DB_PATH. Vectors from different
models are not comparable, even when their dimensions match. After changing RAG_DTYPE or
either prefix option, re-ingest all indexed documents before searching. sync skips unchanged
files.
The CLI does not read MCP client configuration. When sharing an index, use the same model,
RAG_DTYPE, and prefix settings for ingestion and search. A change to RAG_DEVICE alone does
not require a new index.
Search Tuning
The first four settings below apply to both MCP and CLI queries. To give exact terms more
weight, try increasing RAG_HYBRID_WEIGHT and compare results on your own questions. External
reranking is MCP-only.
Variable | Default | Description |
|
| Keyword boost factor (0.0–1.0). 0 disables keyword reranking; 1 applies the maximum boost. |
| (not set) |
|
| (not set) | Filter out low-relevance results (e.g., |
| (not set) | Limit results to top N files (e.g., |
| (not set) | MCP only: external command; |
|
| Time budget per rerank call in milliseconds (100–600000). |
External Reranking (RAG_RERANK_CMD)
The command reads search results, including matched text, from stdin. If it calls a remote service, that text may leave your machine.
Give the executable and its complete argument template. Put {query} and {top} where the
command expects the query and result count. Single or double quotes group paths or arguments
containing spaces, and backslashes stay literal. The server runs the executable without a
shell, so an npm-installed .cmd shim on Windows will not start.
{
"env": {
"RAG_RERANK_CMD": "/path/to/reranker --query {query} --top {top}",
"RAG_RERANK_TIMEOUT_MS": "10000"
}
}The command must read and return results in the format defined by the query output schema. It can remove or reorder results and modify their text. The server returns its output.
Results keep their original order if the command fails, times out, or returns output that does not match the schema.
By default, ingestion indexes only text. To make PDF figures searchable, enable local caption
generation with visual: true in MCP or --visual in the CLI. Captions are generated
descriptions, not OCR or exact transcriptions.
fast (default) downloads about 250 MB on first use. Choose quality for labels and text
within figures; it downloads about 1.7 GB and takes longer to run.
Select the profile with visualQuality: "quality" in MCP or --visual-quality quality in the
CLI.
npx mcp-local-rag ingest ./docs/paper.pdf --visual --visual-quality qualityTo return images with matching text, use STORE_IMAGES=true in MCP or --images with CLI
ingest and sync. This is independent of caption generation and supports detected PDF
figures/tables and supported DOCX PNG/JPEG images.
npx mcp-local-rag ingest ./docs/paper.pdf --imagesSync preserves each PDF's caption profile. CLI sync --visual --visual-quality quality changes
the profile even for unchanged PDFs; MCP sync preserves it. To turn captions off, ingest the
file normally. To retry failed captions, re-ingest with the desired visual profile.
Image storage must be enabled on each ingestion or sync that processes the file. Changing the image setting alone does not refresh unchanged files; re-ingest them to apply it.
Security and Operation
Treat captions and retrieved document text as source material, not instructions.
File access is restricted to
BASE_DIR,BASE_DIRS, or CLI--base-dirroots.Symlinks that resolve outside every configured root are rejected.
Document processing and search make no network requests after the required models are cached, unless
RAG_RERANK_CMDnames a command that makes them.The server is designed for one local user and does not provide authentication or access control.
Do not run multiple CLI or MCP writers against the same
DB_PATH. Read-only queries can run while a sync is active.Re-ingestion can reuse saved embeddings for unchanged text when the model and settings match, avoiding repeated computation. The cache is stored in
DB_PATH/embedding-cache. Deleting it leaves the index searchable, but later ingestion will recompute those embeddings.Back up an index by copying its
DB_PATHdirectory while no writer is active.
"No results found"
Documents must be ingested first. Run "List all ingested files" to verify. If results are
missing after a sync, check that ingestion and search use the same absolute DB_PATH; a
relative path may point to a different index.
Model download failed
Check internet connection. If behind a proxy, configure network settings. The model can also be downloaded manually.
"File too large"
Default limit is 100MB. Split large files or increase MAX_FILE_SIZE.
Slow queries
Check chunk count with status. Large documents with many chunks may slow queries. Consider
splitting very large files.
"Path outside BASE_DIR"
Ensure file paths are within one of the configured roots (BASE_DIR, any BASE_DIRS entry, or
any CLI --base-dir). Use absolute paths.
"BASE_DIRS must be a JSON array..."
BASE_DIRS accepts a JSON array of one or more non-empty path strings:
Valid:
BASE_DIRS='["/Users/me/work","/Users/me/specs"]'Invalid:
BASE_DIRS=/a:/b(delimiter syntax not supported)Invalid:
BASE_DIRS='[]'(empty array)
MCP client doesn't see tools
Verify config file syntax
Restart client completely (Cmd+Q on Mac for Cursor)
Test directly:
npx mcp-local-ragshould run without errors
Contributing
Contributions welcome! See CONTRIBUTING.md for setup and guidelines.
License
MIT License. Free for personal and commercial use.
Blog Posts
Building a Local RAG for Agentic Coding: Technical deep-dive into the semantic chunking and hybrid search design.
Acknowledgments
Built with Model Context Protocol by Anthropic, LanceDB, and Transformers.js.
Available Tools
9 toolsdelete_fileA
Delete a previously ingested file or data from the vector database. Use filePath for files ingested via ingest_file, or source for data ingested via ingest_data. Either filePath or source must be provided. Returns deleted (operation succeeded), removedChunks, and existed (whether anything was actually present).
Inputs: source, filePath.
ingest_dataA
Ingest in-memory content as a string (use ingest_file for files on disk). The source identifier enables re-ingestion to update existing content. Returns { filePath, chunkCount, timestamp, fileTitle }.
Inputs: content (required), metadata (required).
ingest_fileA
Ingest a document file (PDF, DOCX, TXT, MD) into the vector database. Path must be absolute; re-ingesting the same path replaces its existing data. Returns { filePath, chunkCount, timestamp, fileTitle }.
Inputs: filePath (required), visual, visualQuality.
list_filesA
List supported files (PDF, DOCX, TXT, MD) under the configured base directories and whether each is ingested. Returns { baseDirs, files, sources }; sources lists ingested items reported apart from the file scan, chiefly ingest_data content (web pages, clipboard, etc.).
Inputs: scope.
query_documentsA
Search ingested documents with hybrid keyword + semantic matching. Use the returned order as the ranking; score may disagree with it. Each has filePath, chunkIndex, text, fileTitle, score (lower is closer), and source (for ingest_data items). Optional sourceContext provides heading paths and original line or physical PDF page ranges; multiple heading paths mean the chunk crosses sections.
Inputs: query (required), limit, scope.
read_chunk_neighborsA
Read the chunks immediately before and after a query_documents result, in the same document, for more surrounding context. Pass chunkIndex from the result plus exactly one of filePath (ingest_file) or source (ingest_data). Returns the target chunk (isTarget: true) and its neighbors, ascending by chunkIndex; an out-of-range chunkIndex returns []. Optional sourceContext provides heading paths and original line or physical PDF page ranges; multiple heading paths mean the chunk crosses sections. Defaults: before=2, after=2 (max 50 each).
Inputs: chunkIndex (required), after, before, source, +1 more.
statusA
Get index status: { documentCount, chunkCount, memoryUsage (MB), uptime (s), ftsIndexEnabled, searchMode }.
Inputs: no named parameters.
sync_startA
Reconcile the index with the files on disk: ingest new and changed files, leave unchanged files alone, and remove index entries for files that are gone. Each changed PDF is re-ingested with the visual profile ("fast" or "quality") already recorded for it, so a PDF indexed with VLM captions keeps them; a PDF with no recorded profile stays text-only. There is no option to change a profile here — use the CLI (mcp-local-rag sync --visual) to set one, or ingest_file to replace the file, where a normal ingest clears the recorded profile. Stored images are unrelated: STORE_IMAGES applies to whatever this run re-ingests and never makes a file changed. Returns { jobId } without waiting for the run to finish; poll sync_status with that jobId for progress and the final outcome. Only one job is kept, and it is lost when the server process exits.
Inputs: path.
sync_statusA
Get the current or latest sync job record: { jobId, state ("running" | "succeeded" | "failed"), total (null until scanning has counted the files on disk), completed (upserted + skipped + empty; pruned is counted separately), summary { upserted, skipped, empty, pruned }, warnings, error (null unless the job failed) }. An unknown jobId means the job was replaced by a newer one or lost with a previous server process.
Inputs: jobId (required).
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 9 tools
Most tools have clearly distinct purposes: ingest_file vs ingest_data split by source type, query_documents vs read_chunk_neighbors split by search vs context, and delete_file/list_files/sync_start are unique. The only mild overlap is status (index state) vs sync_status (sync job record), but descriptions disambiguate them well.
Seven of nine tools follow a clean verb_noun pattern (query_documents, read_chunk_neighbors, delete_file, ingest_data, ingest_file, list_files, sync_start). The two noun-only names (status, sync_status) are minor deviations but still readable and consistent with each other.
Nine tools is well-scoped for a local RAG server, with each tool earning its place across ingest, query, context, delete, list, sync, and status concerns. No redundant or filler tools.
The lifecycle is well covered: ingest (file/data), query, context retrieval, list, delete, sync, and status. The main gap is that changing a PDF's visual profile is only possible via the CLI, not through an MCP tool, though re-ingest provides a workaround.
Maintenance
Related MCP Connectors
Read and search folders on your own computer with explicit consent. Files are never uploaded.
A search engine that learns from your grades. Store text, records, images or embeddings.
Convert PDFs, Office docs, websites and audio into AI-ready Markdown, then search and ask your docs.
Self-hosted AI workspace: semantic RAG document search, long-term memory, and AI chat via MCP.
Related MCP Servers
- AlicenseAqualityDmaintenanceLocal offline semantic search over documents (txt, md, pdf, docx, pptx, csv). Indexes folders into a LanceDB vector database with multilingual embeddings and supports hybrid vector + keyword search via Reciprocal Rank Fusion. No API keys, no cloud, no Docker required.431AGPL 3.0
- FlicenseAqualityDmaintenanceEnables indexing local documents (PDF, Markdown, text, code) into a knowledge base and querying them via semantic search using local embeddings, all running privately on your machine.4-
- AlicenseNot gradedqualityBmaintenanceSemantic search and retrieval system for local documents using vector embeddings, enabling AI-powered search across your document collections with support for multiple embedding providers.9MIT
- FlicenseNot gradedqualityDmaintenanceEnables semantic search over personal files using natural language, with optional AI summarization, all running locally.-