Skip to main content
Glama
deenesjakoruzh
Ayush-yadav11

quillrag

by Ayush-yadav11
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Rust
Local

quillrag

One file. Zero dependencies. Ready before your editor finishes loading.

A local RAG engine in a single static binary — MiniLM embeddings compiled inside, hybrid dense + BM25 retrieval, MCP-native. No Node, no Python, no model download on first query.

release platforms license

Why quillrag

~20 ms to ready

MCP handshake completes before the model even loads

Zero runtime deps

no Node, no Python, no pip/npm, no model downloads — ever

Hybrid retrieval

dense cosine ⊕ BM25 fused with Reciprocal Rank Fusion

Private by construction

no network code path after installation

One file, three OSes

~105 MB (the model lives inside), CI-built for linux/macOS/Windows

Related MCP server: mcp-fts5-starter

Quick start

# 1. grab a prebuilt binary (or cargo install --path .)
gh release download --repo Ayush-yadav11/quillrag -p '*linux*'
tar xzf quillrag-x86_64-linux.tar.gz && chmod +x quillrag

# 2. point it at any folder of notes/docs/code
./quillrag index ~/notes          # incremental walk

# 3. ask it something
./quillrag search "how does backpropagation work"

Or wire it straight into Claude Desktop / Cursor and let the AI search your notes mid-conversation — config below.

$ ./quillrag serve --data-dir ~/.local/share/quillrag
2026-08-26 INFO quillrag 0.1.2 ready in 41ms      <- handshake-ready before the model loads

Why it's fast

Stage

Cost

Binary start + MCP initialize

~20 ms (measured: store open + tool registration only)

First rag_search / rag_index call

+~300 ms one-time (mmap safetensors, build BERT graph)

Subsequent searches

~25 ms per query (2-core CPU, small corpus)

Re-indexing unchanged corpus

near-zero (FNV content hash skip)

The embedding model is lazy: the MCP handshake and rag_status never touch it, so editors see an instant server.

Install

Download a prebuilt archive from the latest release — Windows x86_64, macOS Apple Silicon, and Linux x86_64 are all built by CI on every version tag:

# linux/macOS example: fetch + extract the latest release
gh release download --repo Ayush-yadav11/quillrag -p '*linux*' | tar xz
chmod +x quillrag && ./quillrag --version

Or build from source:

cargo install --path .

Cross-compile targets used by CI: x86_64-unknown-linux-gnu, aarch64-apple-darwin, x86_64-pc-windows-msvc.

Wire it into your editor

Claude Desktop / Cursor / any MCP client:

{
  "mcpServers": {
    "quillrag": {
      "command": "/usr/local/bin/quillrag",
      "args": ["serve"],
      "env": { "QUILLRAG_DATA": "~/.local/share/quillrag" }
    }
  }
}

Or just run ./quillrag serve and point any stdio client at it.

Tools

Tool

What it does

rag_index

Incrementally index a directory/file. Skips unchanged files, prunes deleted ones, re-embeds only diffs.

rag_search

Hybrid retrieval: dense MiniLM cosine + BM25 keyword, fused with Reciprocal Rank Fusion. Returns ranked chunks with source paths.

rag_status

Document/chunk counts, bytes indexed, file-type breakdown.

rag_clear

Wipe everything.

CLI equivalents (same engine):

quillrag index ~/notes              # incremental walk
quillrag search "auth flow" -k 5    # one-shot search
quillrag status                     # stats
quillrag clear                      # wipe

Design

  • Embeddings: candle (pure Rust) running sentence-transformers/all-MiniLM-L6-v2 — masked mean pooling + L2 norm, numerically matching sentence-transformers on CPU. Weights are include_bytes!-ed into the binary and mmap'd from a materialized cache on first load.

  • Storage: single redb file — chunk text, raw f32 vectors, document metadata. Atomic commits; crash-safe.

  • Keywords: tantivy BM25 sidecar index rebuilt per indexing pass (cheap at pocket scale).

  • Fusion: Reciprocal Rank Fusion (Σ 1/(60+rank)) — no score-scale tuning, robust to heterogeneous rankings.

  • Chunking: paragraph-first with 1000-char cap and 120-char overlap; oversized paragraphs hard-split at sentence boundaries.

File types indexed by default

md markdown txt rst json yaml yml toml csv tsv html htm xml log rs py js jsx ts tsx go c h cpp hpp java rb sh bash zsh sql proto graphql dockerfile makefile ini cfg conf env — extend with -e ext1,ext2 / "extensions": [...].

Ignored dirs: every dot-directory (.git .obsidian .vscode …) plus node_modules target dist build venv __pycache__ vendor.

Privacy & footprint

Everything runs locally: embeddings, storage, search. Nothing leaves the machine — there is no network code path at all after installation.

Binary ≈ 105 MB (the model lives inside). RAM ≈ 120 MB resident while idle, spiking to ~250 MB during batch embedding.

Scaling & limits

quillrag stores everything in a single redb file and runs dense retrieval as an exact, single-threaded linear scan over all vectors — no ANN index yet. That makes the relevant limit query latency, not storage. Storage scales to millions of chunks; retrieval speed is O(N) per query.

Corpus

Vectors

Approx. RAM (f32)

Steady-state query

1K chunks

1K

~1.5 MB

~25 ms (measured)

10K chunks

10K

~15 MB

~250 ms (extrapolated)

100K chunks

100K

~154 MB

~2–5 s (extrapolated)

1M chunks

1M

~1.5 GB

20–60 s (extrapolated — not viable without ANN)

Verified on a corpus of 1K chunks (5/5 tests including real JSON-RPC-over-stdio e2e); figures above 1K are extrapolated from the O(N) dense-scan cost, not measured. A synthetic scale probe (src/bin/quillbench.rs) exists to measure the curve on your own hardware — run cargo build --release && ./target/release/quillbench.

What this means in practice:

  • Great fit: personal/local knowledge bases, project docs, notes, code — up to low-tens-of-thousands of chunks where sub-second-to-interactive latency holds.

  • Away from the sweet spot: corpora in the hundreds of thousands+ where you need interactive (<200 ms) retrieval — you'll want an ANN index (see Roadmap).

How it compares to common alternatives on the relevance axis:

  • Embedding-only (e.g. raw FAISS flat / simple vector store): same all-MiniLM-L6-v2 ceiling as quillrag's dense path, but quillrag adds BM25 + RRF fusion, which wins on keyword-heavy queries (error codes, IDs, exact tokens). quillrag has no reranker or metadata filtering, which llama-index offers on top.

  • llama-index local backends: functionally similar hybrid retrieval (BM25 + vector + RRF). quillrag trades llama-index's rich reranking/parent-child chunking/query-expansion for a zero-dependency single binary and instant startup. Relevance on a standard dataset (BEIR/MS MARCO) is not yet benchmarked — see the open issue tracking ANN + a relevance baseline.

Roadmap

quillrag is deliberately minimal today. The big unlock is an approximate nearest-neighbor index:

  • ANN (HNSW / IVF) over the dense vectors — turns O(N) scan into sub-millisecond ANN lookup, pushing the interactive ceiling from ~10K to millions of chunks on a single machine.

  • Quantization (PQ / SQ) — drops vector RAM from 4 bytes/dim to ~1 byte/dim, so 1M chunks ≈ 380 MB instead of 1.5 GB.

  • Multi-threaded scan — parallelize the current exact path as a stopgap.

  • Reranker hook — optional cross-encoder rerank of the fused top-k.

  • Relevance benchmark — BEIR / MS MARCO nDCG@10 vs. llama-index baselines.

Track the ANN work here: issue #1 — "ANN index for <1M chunks."

FAQ

Is it really one file? Yes. The MiniLM weights + tokenizer are compiled in via include_bytes!. No npm install, no Python, no model download on first query. The binary is ~105 MB because the model lives inside it.

Why is startup so fast? The embedding model is lazy. The MCP handshake and rag_status never touch it — editors see a ready server in ~20 ms. The model only loads on the first rag_search / rag_index (~300 ms one-time).

What's the largest corpus it handles? Verified at 1K chunks (~25 ms/query). The architecture scales to millions of stored chunks; interactive retrieval holds up to low-tens-of-thousands today, and an ANN index (Roadmap) extends that to 1M+.

How is this different from llama-index? Similar hybrid retrieval quality, but quillrag is a single static binary with no runtime/dependency footprint and instant startup. llama-index adds rerankers, sophisticated chunking, and query expansion that quillrag doesn't have yet.

What file types are indexed? md markdown txt rst json yaml yml toml csv tsv html htm xml log rs py js jsx ts tsx go c h cpp hpp java rb sh bash zsh sql proto graphql dockerfile makefile ini cfg conf env — extend with -e.

Does it phone home? No. There is no network code path after installation.

Changelog

  • v0.1.3 — MCP tool descriptions rewritten for clarity, parameter semantics, and behavioral transparency (read-only/destructive flags, usage guidance); server.json shipped in-repo for MCP Registry publishing.

  • v0.1.2 — skip all dot-directories when indexing (.obsidian plugin configs no longer pollute results); first fully automated 3-platform CI release. Upgrade note: run quillrag clear once and re-index.

  • v0.1.1 — CI-built release artifacts for linux/macos/windows with checksums.

  • v0.1.0 — initial public release; renamed from pocketrag.

Development

cargo test                    # unit + end-to-end (spawns real stdio servers)
cargo run -- serve            # dev server
RUST_LOG=debug cargo run ...  # verbose logs (stderr only)

License: MIT

Available Tools

4 tools
rag_clearA

Destructive: permanently deletes ALL indexed documents, chunks, and embeddings from the knowledge base. Cannot be undone — source files on disk are not touched, but re-indexing from scratch is required afterwards. Confirm with the user before calling.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Since there are no annotations, the description carries the full burden of behavioral disclosure. It explicitly states the operation is destructive and permanent, describes what is deleted (documents, chunks, embeddings), what is not deleted (source files on disk), and the necessity of re-indexing. It also includes a confirmation requirement. This is exemplary transparency for a destructive tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the critical warning 'Destructive'. Each sentence adds essential information: scope, permanence, what is preserved, and the confirmation requirement. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, no-output-schema destructive tool, the description covers all necessary aspects: what it does, consequences, side effects, and a required user interaction. An agent has everything needed to call it correctly and safely. The absence of an output schema does not matter since no output is described.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so there is nothing to explain beyond the schema. The baseline for 0-parameter tools is 4, and the description does add relevant context about the scale of the deletion, which is useful. No additional parameter details are needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific verb 'delete' and the resource (all indexed documents, chunks, and embeddings from the knowledge base). It distinguishes this tool from siblings like rag_search, rag_status, and rag_index by emphasizing it is a destructive clear-all operation. The scope is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context (destructive, permanent) and an explicit instruction to confirm with the user. It also notes that re-indexing is required afterwards, which implies it should only be used when a full reset is intended. However, it does not explicitly name alternatives or say when not to use it, but the destructive nature makes the usage context obvious.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rag_indexA

Index local documents into the knowledge base before searching. Pass an absolute directory path for a recursive incremental walk (skips unchanged files by content hash, prunes entries for deleted files) or a single file path. Idempotent and safe to re-run; only changed content is re-embedded. Supported types: md/txt/code files (see README for the full list); dot-directories like .git and .obsidian are skipped automatically. After indexing completes, use rag_search to query. To remove everything instead, use rag_clear.

ParametersJSON Schema
NameRequiredDescriptionDefault
pathYesAbsolute path to a directory of documents (or a single file) to index.
extensionsNoExtra file extensions to include beyond the defaults

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It thoroughly covers the tool's behavior: recursive incremental walk, content-hash change detection, deletion pruning, idempotency, re-embedding only changed content, and automatic skipping of dot-directories. No contradictions with any structured metadata exist.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (four sentences), front-loaded with the primary purpose, and each sentence contributes meaningful information. No redundancy or filler; it efficiently packs behavioral details and sibling references without bloat.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description adequately explains the tool's actions and effects for an indexing operation, given that no output schema exists. It mentions the workflow step ('After indexing completes, use rag_search') but does not specify the response format or return value. This is a minor gap; for a tool like this, the key context is what gets destroyed/created and when to use it, which is well covered.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters. The description adds value by clarifying that 'path' accepts an absolute directory or a single file, and mentions supported types (md/txt/code) which relates to the 'extensions' parameter's defaults. This goes beyond the schema's basic explanation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb ('Index') and resource ('local documents into the knowledge base'), and explicitly distinguishes itself from siblings by noting that rag_search is for querying and rag_clear is for removal. This makes the tool's role unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit usage context: 'before searching', and directs users to use rag_search after indexing and rag_clear for removal. This effectively routes the agent to the correct workflow and alternatives, leaving no ambiguity about when to invoke this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rag_statusA

Report knowledge base statistics: document count, chunk count, total indexed bytes, and per-file-type breakdown. Read-only and instant (does not load the embedding model). Use it to check whether anything is indexed before running a search.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden. It discloses that the tool is read-only, instant, and does not load the embedding model. This is good behavior disclosure for a report tool, though it doesn't mention potential limitations like staleness or exact response format, which is minor given the simplicity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with zero waste. The first sentence lists the precise report contents, and the second gives usage and behavioral context. Information is front-loaded, making it easy to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no parameters and no output schema, the description fully covers what it does, what it returns (stats and breakdown), and when to use it. Nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline is 4. The description doesn't need to explain parameters; it appropriately stays silent on them. No additional meaning is required beyond what the schema (empty) provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool reports knowledge base statistics (document count, chunk count, total indexed bytes, per-file-type breakdown). It uses a specific verb and resource, and the content distinguishes it from sibling tools like rag_search, rag_clear, and rag_index.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly says to use it 'to check whether anything is indexed before running a search,' providing a clear context of use. It doesn't explicitly list when not to use it or alternatives, but the intended use case is clear and the siblings are evident from the tool names.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv0.1.3
    • First observedrag_clear
    • First observedrag_index
    • First observedrag_search
    • First observedrag_status
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A4.6/5.0

Scored across 4 tools

Disambiguation5/5

Each tool has a single, clear responsibility: search, status, index, and clear. There is no overlap—even search and status are distinguished by read-only purpose versus metadata reporting. The descriptions explicitly note when to prefer one over the other.

Naming Consistency5/5

All tools follow the same 'rag_' prefix followed by a lowercase verb or noun, all in snake_case. The pattern is uniform and predictable: rag_search, rag_status, rag_clear, rag_index. No mixed conventions or stylistic deviations.

Tool Count5/5

With 4 tools, the server is well-scoped for a RAG knowledge base service. It covers the core operations without unnecessary bloat or missing essentials. This is within the ideal 3-15 range and each tool earns its place.

Completeness4/5

The tools cover the full lifecycle: index (create/update), search (read), status (read metadata), and clear (delete). The only minor gap is the lack of a selective document deletion, but incremental indexing and pruning handle updates well, so agents can work around this limitation.

Maintenance

ActivityMaintained
ResponsivenessWithin a week

Resources

Related MCP Connectors

Related MCP Servers

  • ragi

    SusuTawar
    A
    license
    A
    quality
    D
    maintenance
    Local-first RAG indexing and semantic search MCP server. Enables document retrieval and context-aware queries using local embedding models.
    3
    6 npm
    MIT

  • mcp-fts5-starter

    zx22413
    A
    license
    Not graded
    quality
    C
    maintenance
    Drop-in MCP server template with SQLite FTS5 search backend. ~300 lines, no vector DB, no embedding API, runs on a Pi.
    MIT

  • corpus-rag

    monahand1023
    A
    license
    Not graded
    quality
    A
    maintenance
    MCP server for local RAG over personal notes, PDFs, and documents, enabling plain-English querying and hybrid search with multi-hop context expansion.
    MIT

  • minirag

    mck-s
    A
    license
    Not graded
    quality
    C
    maintenance
    MCP server for a self-hosted RAG system that enables AI tools to search and retrieve grounded answers from locally ingested documents via MCP tools, with local embeddings and no API key required.
    MIT