Skip to main content
Glama
docsonar

docsonar

by ribhav-jain

docsonar

PyPI CI Python License: MIT

Chat with your folders. A local document search MCP server — your files never leave your machine.

docsonar indexes folders of documents into a single SQLite database and gives any MCP client (Claude Desktop, Claude Code, and others) hybrid keyword + semantic search over them. Fully offline, zero configuration, read-only by design.

Quickstart

Claude Code

Add to your project's .mcp.json (or ~/.claude.json for all projects):

{
  "mcpServers": {
    "docsonar": {
      "command": "uvx",
      "args": ["docsonar"]
    }
  }
}

Claude Desktop

Add to claude_desktop_config.json (Settings → Developer → Edit Config):

{
  "mcpServers": {
    "docsonar": {
      "command": "uvx",
      "args": ["docsonar"]
    }
  }
}

Running from a source checkout instead: "command": "uv", "args": ["run", "--directory", "/path/to/docsonar", "docsonar"].

Then just talk to it: "Add my ~/Documents/notes folder and find everything about quarterly planning." The first index downloads the embedding model (~130 MB, one time); keyword search works immediately while that happens.

For HTTP instead of stdio: uvx docsonar --transport http --port 8365.

Related MCP server: local-document-rag-agent

Tools

Tool

Purpose

add_folder(path, include_globs?, exclude_globs?)

Register a folder; indexing starts in the background

remove_folder(path)

Unregister and purge its index data

list_folders()

Registered folders with counts and last index time

search(query, top_k?, folder?, file_type?, mode?)

Hybrid keyword+semantic search (RRF-fused); ranked passages with file path, heading/page location, and snippet

read_file(path, start_line?, end_line?)

File text (extracted text for pdf/docx/html); refuses paths outside registered folders

find_similar(path, top_k?)

Documents most similar in meaning to a given file

reindex(folder?, force?)

Incremental refresh: new/changed files reindexed, deleted files purged

index_status()

Index totals, embedding model state, background job progress, failed files

Supported formats: txt, md, pdf (page-aware, cites p. 12), docx, html.

Architecture

flowchart LR
    Client["MCP client<br/>(Claude Desktop / Code)"] <-->|stdio or HTTP| Server["FastMCP server<br/>8 tools"]
    Server --> Sec["path security<br/>(registered folders only)"]
    Server --> Search["hybrid search<br/>BM25 + cosine KNN + RRF"]
    Server --> Worker["background indexer<br/>(worker thread + queue)"]
    Worker --> Parsers["parsers<br/>txt · md · pdf · docx · html"]
    Parsers --> Chunker["heading/page-aware chunker<br/>~400 tokens, 60 overlap"]
    Chunker --> DB[("SQLite<br/>FTS5 + sqlite-vec")]
    Search --> DB
    Embed["sentence-transformers<br/>bge-small-en-v1.5 (local)"] --> Worker
    Embed --> Search

Every chunk stores its location (heading path like Setup > Windows, or PDF page range), so search results can cite report.pdf, p. 12.

Design decisions

  • Read-only by design. There are no write, move, or delete tools, and there never will be. The server only reads files inside folders you explicitly register — with symlink-escape protection and strict path resolution — so the blast radius of a misbehaving client is zero.

  • SQLite as the single store. FTS5 gives production-grade BM25 keyword search in the standard library, and sqlite-vec puts vectors in the same file. One database file, no services to run, trivial to back up or delete.

  • Hybrid search by default. BM25 and cosine-KNN rankings are fused with reciprocal-rank fusion (k=60). Exact identifiers and rare terms win on the keyword side; paraphrased questions win on the semantic side; RRF needs no score calibration between them. mode lets the caller force either side.

  • Local embeddings. BAAI/bge-small-en-v1.5 (384-dim) — same size class as the classic all-MiniLM-L6-v2 but stronger on retrieval benchmarks. Downloads on first index, runs on CPU, lazy-loaded so server startup stays instant. If the model can't load, everything degrades gracefully to keyword search and tool responses say so.

  • A tool surface built for an LLM caller. search returns enough per hit (path, location, score, snippet) to decide what to read next without another round trip; results carry stable chunk_ids; every degradation is reported in-band via note/embedding_note fields instead of failing. There's no "answer the question" tool on purpose — the calling model does the reasoning; docsonar does retrieval.

  • Incremental by content, not just mtime. Reindexing checks mtime+size first, then falls back to a SHA-256 content hash — touched-but-identical files are skipped, and deleted files are purged from both the FTS and vector indexes.

  • Scanned PDFs fail loudly. A PDF with no extractable text is reported as failed with a clear reason rather than silently indexed as empty. OCR is out of scope.

Benchmarks

Synthetic corpus of 200 markdown files (600 chunks); Intel Core Ultra 5 125U (laptop, CPU-only), Windows 11, Python 3.12. Reproduce with uv run python scripts/benchmark.py.

Operation

Result

Index, keyword-only

0.7 s (≈280 files/s)

Index, with embeddings

59 s (≈3.4 files/s — embedding-bound)

Incremental reindex, nothing changed

0.03 s

Search, keyword

8.9 ms median

Search, semantic

35 ms median

Search, hybrid

54 ms median

Embedding model load (once per process)

~21 s

Database size

2.6 MB (1.1 MB keyword-only)

Configuration

Zero config needed. To customize, create config.toml in the platform config directory (Windows: %LOCALAPPDATA%\docsonar\, macOS: ~/Library/Application Support/docsonar/, Linux: ~/.config/docsonar/):

embedding_model = "BAAI/bge-small-en-v1.5"  # any sentence-transformers model
chunk_target_tokens = 400
chunk_max_tokens = 512
chunk_min_tokens = 100
chunk_overlap_tokens = 60
max_file_size_mb = 50
extra_ignore_dirs = ["Archive"]

CLI flags: --transport stdio|http, --host, --port, --db-path, --config.

The index database lives in the platform data directory (Windows: %LOCALAPPDATA%\docsonar\, macOS: ~/Library/Application Support/docsonar/, Linux: ~/.local/share/docsonar/).

Running from source

Requires uv (it installs the right Python automatically):

# Windows
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
# macOS / Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

git clone https://github.com/ribhav-jain/docsonar.git
cd docsonar
uv sync --dev    # creates .venv and installs everything, incl. dev tools

Run the tests

uv run pytest                          # full suite (~2 s, no model download needed)
uv run pytest -v                       # verbose, one line per test
uv run pytest tests/test_search.py     # one file
uv run pytest -k incremental           # tests matching a keyword
uv run ruff check .                    # lint
uv run mypy src tests                  # strict typecheck
uv run python scripts/benchmark.py     # perf numbers (downloads the real model)

Run the server

uv run docsonar                                  # stdio — for MCP clients (Claude Desktop/Code)
uv run docsonar --transport http --port 8365     # HTTP — for manual testing (Postman, curl)

VS Code: press F5 — launch configs for the HTTP server and the test suite are in .vscode/launch.json.

Try the tools without an MCP client

With the HTTP server running, the endpoint is http://127.0.0.1:8365/mcp. Recent Postman versions can connect directly (New → MCP Request, transport HTTP) and show all 8 tools as forms. For raw HTTP, MCP is JSON-RPC over POST with two setup calls, then tool calls. Send every request with headers Content-Type: application/json and Accept: application/json, text/event-stream:

// 1. initialize — copy the `mcp-session-id` RESPONSE header and send it
//    back as a request header on every call below
{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"postman","version":"1.0"}}}

// 2. initialized notification (expect 202, empty body)
{"jsonrpc":"2.0","method":"notifications/initialized"}

// 3. list the tools
{"jsonrpc":"2.0","id":2,"method":"tools/list"}

Then call tools — the repo ships a sample corpus in examples/sample-docs to play with (use forward slashes in JSON to avoid escaping; they work on Windows too):

// Register the sample folder (first ever call downloads the embedding model, ~30 s)
{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"add_folder",
  "arguments":{"path":"C:/path/to/docsonar/examples/sample-docs"}}}

// Watch indexing progress until indexing.active is false
{"jsonrpc":"2.0","id":4,"method":"tools/call","params":{"name":"index_status","arguments":{}}}

// Hybrid search — finds the expense-policy PDF, cites its page
{"jsonrpc":"2.0","id":5,"method":"tools/call","params":{"name":"search",
  "arguments":{"query":"meal reimbursement limit","top_k":3,"mode":"hybrid"}}}

// Semantic search — no keyword overlap needed
{"jsonrpc":"2.0","id":6,"method":"tools/call","params":{"name":"search",
  "arguments":{"query":"how hot should water be for green tea","mode":"semantic"}}}

// Read a hit (works on PDFs too — returns extracted text)
{"jsonrpc":"2.0","id":7,"method":"tools/call","params":{"name":"read_file",
  "arguments":{"path":"C:/path/to/docsonar/examples/sample-docs/expense-policy.pdf"}}}

// "More like this"
{"jsonrpc":"2.0","id":8,"method":"tools/call","params":{"name":"find_similar",
  "arguments":{"path":"C:/path/to/docsonar/examples/sample-docs/espresso-guide.md","top_k":3}}}

// Refresh after files change on disk (force:true rebuilds everything)
{"jsonrpc":"2.0","id":9,"method":"tools/call","params":{"name":"reindex","arguments":{}}}

// Clean up — unregisters the folder and purges its index data
{"jsonrpc":"2.0","id":10,"method":"tools/call","params":{"name":"remove_folder",
  "arguments":{"path":"C:/path/to/docsonar/examples/sample-docs"}}}

Security check worth trying: read_file with any path outside a registered folder (e.g. C:/Windows/System32/drivers/etc/hosts) returns a refusal, not file content.

CI runs lint, typecheck, and the test suite on Linux, macOS, and Windows.

License

MIT

Available Tools

8 tools
add_folderA

Register a local folder so its documents become searchable.

Use this once per folder before searching it. Indexing runs in the background — this returns immediately with the file count found; call index_status to track progress (the first ever index also downloads the embedding model, ~130 MB). path must be an absolute path to an existing directory. Optional globs match paths relative to the folder using forward slashes (e.g. include_globs=["docs/**/.md"], exclude_globs=["drafts/"]); omit both to index every supported file. Supported types: txt, md, pdf, docx, html.

Returns {ok, folder, files_found, status: "indexing"}. files_found counts supported files matching the globs; unsupported files are silently ignored. Searches issued while indexing runs work on the portion indexed so far.

Inputs: path (required), exclude_globs, include_globs. Returns: object

find_similarA

Find indexed documents most similar in meaning to a given file.

Use for "more like this" — e.g. finding related notes, duplicate content, or other documents on the same topic. path must be a file inside a registered folder that has already been indexed. Similarity uses stored embeddings (whole-file centroid vs. best-matching chunks), so it works even if the embedding model is not currently loaded.

Returns {ok, source, similar: [{path, file_type, score, preview}, ...]}. score is cosine similarity (0-1, higher is more similar); preview is the best-matching section's heading or opening text. Returns an error if the file has no stored embeddings.

Inputs: path (required), top_k. Returns: object

index_statusA

Report index health and background indexing progress.

Use after add_folder or reindex to check whether indexing has finished, or to diagnose why search results look incomplete (failed files, embeddings unavailable). Cheap to call.

Returns {ok, db_path, db_size_bytes, embedding: {model, status}, totals: {folders, files_indexed, files_failed, chunks, chunks_embedded}, indexing: {active, current: {folder, files_done, files_total} | null, queued_folders, recent_jobs}, failed_files: [{path, error}, ...] (up to 10)}. embedding.status is "ready", "not loaded yet ...", or "error: ..." — errors mean keyword-only search.

Inputs: no named parameters. Returns: object

list_foldersA

List registered folders with file/chunk counts and last index time.

Use this to check what is searchable before calling search, or to find the exact registered path for remove_folder. Returns {ok, folders: [{path, files, failed_files, chunks, added_at, last_indexed_at}, ...]}.

Inputs: no named parameters. Returns: object

read_fileA

Read the text of one indexed document, optionally a line range.

Use after search to pull full context around a hit. path must lie inside a registered folder — anything else is refused (this server never reads outside registered folders). Plain-text formats (txt, md) return the raw file; binary formats (pdf, docx, html) return the extracted text, paragraphs separated by blank lines. start_line/end_line are 1-based and inclusive; omit both for the whole file. Output is capped at ~100k characters; truncated is true if the cap hit, in which case request a narrower line range.

Returns {ok, path, start_line, end_line, total_lines, truncated, text}.

Inputs: path (required), end_line, start_line. Returns: object

reindexA

Refresh the index: pick up new, changed, and deleted files.

Use when documents changed on disk since they were indexed. Runs in the background; track with index_status. Incremental by default — unchanged files (by mtime, then content hash) are skipped, files deleted from disk are purged from the index. folder limits the refresh to one registered folder (path as shown by list_folders); omit it to refresh all. force=true rebuilds every file from scratch regardless of change detection (use after changing chunking or embedding settings).

Returns {ok, queued: [folder paths], force}.

Inputs: force, folder. Returns: object

remove_folderA

Unregister a folder and purge all its index data (files, chunks, search index).

The documents on disk are untouched — this only forgets them. Any in-progress background indexing of this folder is cancelled. Use list_folders to see what is registered. Returns {ok, folder, files_purged}.

Inputs: path (required). Returns: object

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4.7/5.0

    Scored across 8 tools

    Disambiguation5/5

    Each tool has a clearly distinct purpose: status reporting, folder listing, searching, similarity lookup, file reading, folder removal/addition, and reindexing. No two tools overlap in a way that would confuse an agent.

    Naming Consistency4/5

    Most tools follow a snake_case verb_noun pattern (list_folders, add_folder, remove_folder, read_file), but a few deviate: 'search' and 'reindex' are single verbs, and 'find_similar' uses an adjective. The style is uniform and readable, but not perfectly consistent.

    Tool Count5/5

    Eight tools is ideal for this document indexing/search server. The count is neither too thin nor bloated, and each tool addresses a necessary part of the workflow.

    Completeness5/5

    The tool surface covers the full lifecycle: folder registration (add_folder, remove_folder), inventory (list_folders), index health (index_status, reindex), and content access (search, find_similar, read_file). No significant gaps exist for the stated purpose.

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      Local MCP server that indexes folders of documents into a hybrid vector + keyword search index for Claude Desktop, with support for PDFs, Office files, and images via OCR.
      MIT
    • A
      license
      Not graded
      quality
      D
      maintenance
      A local document search MCP server that indexes personal files and enables hybrid search (BM25 + semantic) for Claude to find and cite internal materials.
      MIT
    • A
      license
      A
      quality
      D
      maintenance
      MCP server that indexes Claude.ai chats and local Claude Code sessions, enabling semantic and keyword search across all your conversations with Claude.
      6
      56 PyPI
      5
      MIT