knowledge-rag is a local-first, MCP-native RAG server that lets AI agents (Claude Code, Cursor, Windsurf, etc.) index and search a private document corpus entirely on-premise — no cloud required.
Search the corpus — search_knowledge runs hybrid semantic + BM25 retrieval with cross-encoder reranking, tunable via hybrid_alpha (keyword ↔ semantic), category filter, min_score threshold, snippet vs. full-chunk mode, and a search_method dispatch selector (auto / hybrid / fts5).
Read documents — get_document returns a full document by filepath (search returns chunks only); search_similar finds thematically related docs from a reference file.
Browse the index — list_documents (optionally by category), list_categories with per-category counts, and get_index_stats for corpus size, embedding model, BM25/cache/watcher health.
Ingest and mutate content — add_document (raw text), add_from_url (fetch, sanitize, convert to markdown, index), update_document (full replacement), remove_document (unindex, optionally delete the file from disk). Changes are searchable immediately.
Rebuild and repair — reindex_documents supports smart incremental (force), nuclear rebuild (full_rebuild), and resume from a checkpoint; poll progress with get_reindex_status.
Measure retrieval quality — evaluate_retrieval reports MRR@5, Recall@5 and Precision@5 against expected-source test cases.
Operate it as a service — stdio for single clients or SSE / streamable-http for shared multi-client access, with bearer-token auth, rate limiting, Prometheus /metrics, /health probes, and opt-in structured JSON logging.
Parse 35 file formats locally — Markdown, PDF, Office docs, notebooks, JSON/CSV/XML, and many code/IaC languages, embedded via in-process FastEmbed ONNX (CPU or optional CUDA/DirectML).
Hybrid search · Cross-encoder reranking · 35 file formats · 100% local · Zero cloud · Enterprise-grade plumbing built-in.
pip install knowledge-rag → restart Claude Code → search_knowledge("your query")Quick Start · Why knowledge-rag · Compare · Enterprise Features · Docs
Chart updated daily by GitHub Action
Related MCP server: nexus-mcp-ci
Most RAG frameworks fall into one of three traps: (1) they require you to ship your data to a cloud API, (2) they hand you 300 building blocks and 0 opinionated defaults, or (3) they bundle RAG as a 5% feature of a much bigger platform you didn't ask for.
knowledge-rag is an MCP-native local RAG server for clients such as Claude Code, Cursor, Windsurf, VS Code, Cline, Gemini CLI and Zed. It includes optional bearer authentication, Prometheus metrics, rate limiting, health probes, structured JSON logging and staging-based rebuilds. Configure the transport and access controls for the intended deployment.
Embedding generation, indexing, and retrieval run locally. Model downloads and add_from_url use network access, and configured MCP clients receive retrieved content. Choose the corpus, client permissions, and network exposure to fit your deployment; this project does not certify regulatory compliance.
pip install knowledge-rag → restart your MCP client → done. No Docker mandatory. No Ollama required. No separate embedding server. Everything runs in-process via FastEmbed ONNX. Works offline after the first model download.
7-pillar quality workflow, a 9-cell OS×Python CI matrix (Linux + Windows + macOS × 3.11/3.12/3.13), and nightly chaos, bounded-memory soak and mutation tests. Inspect the checks on the revision being deployed; CI coverage is not a guarantee of zero defects.
No token bills. No SaaS tier. No paid features hidden behind a wall. MIT license, forever. Runs on the laptop you already have — GPU optional, CPU works fine with FastEmbed ONNX.
Compare the features needed for your deployment rather than treating framework checklists as a benchmark. Knowledge-rag combines MCP tools, local embeddings, ChromaDB persistence, BM25, optional FTS5 and optional reranking in one process.
Decision | What to verify |
Local or remote processing | Where models run, which client receives content, and which features need network access |
Retrieval quality | Expected-source queries on your corpus, including its languages and identifiers |
Performance | Cold initialization, indexing, uncached queries, memory peaks and concurrent use on comparable hardware |
Operations | Authentication, metrics exposure, backups, recovery and one-writer ownership |
Extensibility | Supported parsers, model profiles and the public MCP contract |
Pick your integration path — knowledge-rag ships the same server through every channel.
pip install knowledge-rag
knowledge-rag init # scaffolds config.yaml + documents/Drop your PDFs, markdown, code files into documents/. Restart your MCP client. Ask it:
search_knowledge("your query")That's it. First query loads the ONNX embedding model (~200MB, one-off download). Subsequent queries are cached and hit sub-second latency.
# config.yaml
server:
transport: "sse" # or "streamable-http"
host: "0.0.0.0"
port: 8179
auth:
bearer_token: "your-secret-token"
rate_limit:
enabled: true
requests_per_minute: 60
metrics:
enabled: true
port: 9179
logging:
format: "json" # ELK / Loki / Datadog / CloudWatch readyknowledge-rag --transport sseHealth probe: curl http://your-host:8179/health → 200 + JSON payload
Prometheus scrape: http://your-host:9179/metrics
MCP dispatcher: authenticated via Authorization: Bearer your-secret-token
stdio is ideal when one MCP client owns one knowledge-rag process. If several clients (e.g. Claude, Cursor, LM Studio, or multiple terminal windows) should share one index and one running process, run knowledge-rag once with Streamable HTTP and point each HTTP-capable client at the same endpoint (prompted by #217 and issue #223):
knowledge-rag --transport streamable-http
# MCP endpoint: http://127.0.0.1:8179/mcpLocal MCP clients (Cursor, LM Studio, etc.): Point their remote-server URL directly at http://127.0.0.1:8179/mcp.
Claude Custom Connectors (Remote MCP):
Network requirement: Custom connector requests originate from Anthropic's cloud infrastructure across web, desktop, and mobile, so they cannot reach local 127.0.0.1. You must expose your server via a secure, publicly accessible HTTPS endpoint (e.g. Cloudflare Tunnel, ngrok, or reverse proxy) at https://your-public-domain/mcp. Configure the proxy to send an upstream Host accepted by the HTTP app for server.host (for the default loopback configuration, use 127.0.0.1:8179); forwarding the public hostname unchanged can produce 421 Invalid Host header. Keep bearer authentication and HTTPS enabled. Local claude_desktop_config.json uses command/args for local stdio processes and does not support remote HTTP endpoints.
Setup & Auth: In Claude (Pro/Max plans: Customize > Connectors > + Add > Add custom connector; Team/Enterprise owners: Organization settings > Connectors), enter your public endpoint. In Request headers, set header name Authorization with value Bearer <your-token> matching server.auth.bearer_token in config.yaml (see Anthropic Custom Connectors documentation).
Transport | Clients | Shared index / process | RAM overhead | Process management | Startup & query latency | Best fit |
| 1 | No | 1 runtime per client; models load lazily in each process when queried | Client manages its own process; no separate server | Spawns per client; pays preflight and first-query model load | Simplest local setup (single client) |
| Many | Yes | Shared server process; single shared model load across clients | Operator manages one background server | Reuses warm server; first-query model load still applies | Legacy HTTP clients |
| Many | Yes | Shared server process; single shared model load across clients | Operator manages one background server | Reuses warm server; first-query model load still applies | Preferred shared-server setup |
When an HTTP transport is selected, knowledge-rag automatically enables its single-instance lock (KNOWLEDGE_RAG_SINGLE_INSTANCE=1) to prevent multiple servers from binding the same data directory or port.
Security note: The default host is
127.0.0.1. If you bind to0.0.0.0or deploy across a network, configureserver.auth.bearer_tokeninconfig.yamland terminate TLS (HTTPS) via a reverse proxy (e.g. Caddy, Nginx) to protect bearer tokens in transit; otherwise the server warns that the HTTP endpoint is unauthenticated.
docker pull ghcr.io/lyonzin/knowledge-rag:latest
docker run -v $(pwd)/documents:/app/documents -p 8179:8179 ghcr.io/lyonzin/knowledge-rag:latestFull installation guide with all 5 methods, 8 MCP client configurations, and GPU setup: docs/INSTALLATION.md →
Installing knowledge-rag gives your agent 13 MCP tools. It does not tell the agent when to use them. That is what the skills/ folder solves — drop-in behavioural skills for Claude Code, Cursor, Windsurf, Cline, Zed, VS Code Copilot that turn "AI with access to RAG" into "AI that actually uses RAG first".
10 skills, MIT licensed, organized by kind:
# | Skill | What it does |
1 | Search the corpus before answering any technical claim | |
2 | Every claim ships with | |
3 | First interaction of a session probes what is indexed | |
4 | 3-step drill: | |
5 | Only hit the web when local RAG comes back empty | |
6 | Bug / error → RAG first for prior fixes | |
7 | Review consults ADRs / patterns before commenting | |
8 | After a decision, index it back — close the feedback loop | |
9 | Security tasks: MITRE / CVE / runbook first | |
10 | Weekly checkup — MRR@5 · Recall@5 · Precision@5 |
Install — pick the shortest path for your machine:
# Option 1 — Via skills.sh (needs Node — one command, zero clone)
npx skills add lyonzin/knowledge-rag
# Option 2 — Via our install.sh (no Node needed; works on Linux/macOS/WSL/Git Bash)
curl -fsSL https://raw.githubusercontent.com/lyonzin/knowledge-rag/master/skills/install.sh | bashBoth restart-Claude-Code and you are done. Option 2 supports --project, --only rag-check-first,rag-cite-sources, --dry-run, --help.
For Cursor, Windsurf, Cline and full manual instructions → skills/README.md · Full catalog with skill chains → skills/CATALOG.md
Once installed, your AI agent gets these 13 tools automatically:
Tool | Purpose |
| Hybrid semantic + BM25 with cross-encoder rerank |
| Retrieve full content of one document |
| Find documents similar to a reference |
| Measure MRR@5 · Recall@5 · Precision@5 |
| Index a new document via MCP |
| Re-index a changed document |
| Drop a document + all its chunks |
| Fetch, sanitize, and index a URL |
| Enumerate indexed documents |
| Auto-tagged by folder path |
| Corpus size, cache hit rate, embedding dim |
| Smart incremental OR nuclear rebuild |
| Live progress polling (async reindex) |
Full API reference with parameter details, return schemas, examples: docs/API.md →
Every RAG framework claims "production-ready." Here is what knowledge-rag ships in the OSS core, verified by regression tests, that competitors either paywall, plugin-ify, or simply don't have.
Bearer token auth on SSE / HTTP transports — constant-time comparison (hmac.compare_digest), RFC 6750 challenge, 401 fenced with WWW-Authenticate header
Path traversal + symlink escape defenses — validate_path_within guarding 6 CRUD tools (CWE-22, CWE-59)
External-content handling — known-token neutralization and a stored provenance fence, with parser metadata propagation. These are limited mitigations; clients must still treat retrieved content as untrusted data. See SECURITY.md.
OpenSSF Best Practices project #13864 · CodeQL weekly scan · Bandit + Semgrep + Gitleaks + pip-audit workflows. Review actual check results and exceptions for the release you deploy.
PyPI Trusted Publishing via OIDC (zero long-lived tokens in CI)
Prometheus /metrics endpoint — custom histogram buckets tuned for RAG (p95 ≤ 10ms fast-path targets), 7 canonical metrics via @instrument decorator on all 13 tools
Rate limiting — thread-safe sliding-window counter, per-client RPM + burst, zero overhead when disabled
Health probes — GET /health and /healthz returning {status, version, uptime_seconds, cache} in front of the auth middleware (probes always succeed)
Structured JSON logging — opt-in via server.logging.format: "json", one JSON object per record ready for ELK / Loki / Datadog / CloudWatch
Public benchmark dashboard on GitHub Pages
SSE / streamable-http transport — 1 server serves N MCP clients, ChromaDB WAL mode enabled automatically, shared embedding model + query cache
BM25 inverted index — keyword candidate retrieval without scoring every document for every query
FTS5 SQLite fast path — opt-in lexical retrieval; measure relevance and latency on your identifiers and corpus
Cross-encoder reranking — Xenova/ms-marco-MiniLM-L-6-v2; optional model scoring of a candidate pool
NVIDIA CUDA embeddings and experimental Windows AMD DirectML opt-in, with actual-session checks and CPU fallback; setup and compatibility limits
Query cache — LRU + 5-minute TTL, with mutation invalidation
Staging rebuild — populate and validate a replacement while the current collection remains available; handled publication failures roll back
Async background reindex with get_reindex_status() polling
Nightly chaos injection — HuggingFace Hub offline · ONNX zero-byte replay · watchdog crash recovery (3 scenarios in tests/chaos/)
50,000-iteration soak test — checks a fixed memory-growth budget with stub embeddings (KNOWLEDGE_RAG_SOAK_ITERATIONS=50000); this is not a one-hour real-model endurance test
Mutation testing (mutmut) on instance_lock + preflight — catches tests that are too weak
Determinism check — full test suite × 3, catches flakes
Backwards compatibility — 13 MCP tools, signature regression tests and legacy YAML fixtures; optional additions preserve existing calls
API surface AST diff — checks structural signature changes alongside behavioral tests
9-cell CI matrix — Linux + Windows + macOS × 3.11 + 3.12 + 3.13
Preset: cybersecurity.yaml · 8 categories · 200+ routing keywords · 69 query expansions
Ingest MITRE ATT&CK, threat reports, exploit writeups and incident reports. Search from an MCP client with search_knowledge("privilege escalation windows"). Offline operation requires cached models; the chosen client receives retrieved content and determines what happens to it afterwards.
Preset: developer.yaml · 9 categories · 150+ routing keywords · 50+ expansions
Replace Confluence hunting. Ingest architecture docs, ADRs, runbooks, code, API specs. Devs ask their AI agent "how do we authenticate the payment service" and get the exact ADR + implementation file citation.
Preset: research.yaml · 9 categories · 100+ routing keywords · 40+ expansions
Index arXiv papers, lab notebooks, dataset documentation. Semantic search finds papers by intent, not just keywords — cross-encoder reranking surfaces the actually-relevant one instead of five that share a term.
Preset: general.yaml · blank slate, pure semantic search
Deploy via SSE on a single VM. 40+ users authenticated via bearer token, rate-limited, Prometheus-monitored, /health probes wired to your load balancer, JSON logs shipped to Datadog. No cloud calls. Meets LGPD, GDPR, HIPAA data-locality requirements by design.
Verified at scale: production reproduction on a 5 889-doc / 75 016-chunk corpus with concurrent queries during a nuclear rebuild — zero downtime, zero errors (see CHANGELOG v4.8.3).
End-to-end view of how MCP clients, the retrieval pipeline, storage, and enterprise plumbing connect. Every arrow is a real code path — nothing pictured here is aspirational.
flowchart TB
subgraph CLIENTS["MCP Clients (any of these)"]
C1[Claude Code]
C2[Claude Desktop]
C3[Cursor]
C4[Windsurf]
C5[VS Code · Cline · Gemini CLI · Zed]
end
subgraph TRANSPORT["Transport Layer"]
T1[stdio<br/>1 process per client]
T2[SSE / streamable-http<br/>1 server serves N clients]
end
subgraph MIDDLEWARE["ASGI Middleware Chain (HTTP mode)"]
M1[HealthMiddleware<br/>/health · /healthz]
M2[BearerAuthMiddleware<br/>constant-time compare]
M3[Rate Limiter<br/>sliding window]
end
subgraph MCP["13 MCP Tools (frozen contract)"]
MT1[search_knowledge]
MT2[get_document · search_similar]
MT3[add_document · add_from_url · update · remove]
MT4[reindex_documents · get_reindex_status]
MT5[list_documents · list_categories · get_index_stats · evaluate_retrieval]
end
subgraph SEARCH["Retrieval Pipeline"]
R[Query Router<br/>lexical vs semantic]
F[FTS5 Fast-Path<br/>opt-in lexical retrieval]
BM[BM25 Inverted Index<br/>keyword candidates]
SE[Semantic Search<br/>FastEmbed ONNX lazy-loaded]
RRF[Reciprocal Rank Fusion]
CE[Cross-Encoder Rerank<br/>MiniLM-L-6-v2]
QC[Query Cache<br/>LRU + 5-min TTL]
end
subgraph STORAGE["Storage (100% local)"]
CH[ChromaDB<br/>vectors + metadata<br/>WAL mode]
FT[SQLite FTS5<br/>lexical index<br/>WAL + busy-timeout]
MD[index_metadata.json<br/>durable state]
end
subgraph INGEST["Document Ingestion"]
FS[documents/ folder]
WD[Watchdog<br/>10s debounce]
PA[Format parsers<br/>MD · PDF · DOCX · code · IaC · IPYNB]
CK[Chunker<br/>markdown-aware · code-aware]
EM[FastEmbed ONNX<br/>384D bge-small-en-v1.5]
DD[SHA256 Dedup]
SW[Staging Rebuild<br/>rollback on handled publication failure]
end
subgraph OBS["Enterprise Observability (opt-in)"]
PM[Prometheus /metrics<br/>7 canonical + histograms]
LG[Structured JSON logs<br/>ELK · Loki · Datadog · CloudWatch]
HC[Health payload<br/>version · uptime · cache stats]
end
subgraph CFG["Configuration"]
YM[config.yaml<br/>+ 5 domain presets]
end
C1 & C2 & C3 & C4 & C5 -->|MCP protocol| T1
C1 & C2 & C3 & C4 & C5 -.->|remote deploy| T2
T1 --> MCP
T2 --> M1 --> M2 --> M3 --> MCP
MT1 --> QC
QC -->|cache miss| R
R -->|lexical| F
R -->|semantic| SE
R -->|hybrid| BM
F --> CH
F --> FT
BM --> CH
SE --> CH
BM --> RRF
SE --> RRF
RRF --> CE
CE --> QC
MT2 --> CH
MT3 --> INGEST
MT4 --> SW
MT5 --> CH
FS --> WD --> PA
PA --> CK --> EM --> DD --> CH
SW -.->|validated publication| CH
SW -.-> FT
CH -.-> MD
MCP -.->|instrumented| PM
MCP -.->|logs| LG
M1 --> HC
YM -.-> SEARCH
YM -.-> STORAGE
YM -.-> OBS
YM -.-> MIDDLEWARE
classDef client fill:#3776AB,stroke:#1e5a8a,color:#fff
classDef transport fill:#00A67E,stroke:#006e54,color:#fff
classDef middleware fill:#6b46c1,stroke:#4c1d95,color:#fff
classDef storage fill:#4b5563,stroke:#1f2937,color:#fff
classDef obs fill:#dc2626,stroke:#7f1d1d,color:#fff
classDef ingest fill:#f59e0b,stroke:#78350f,color:#fff
class C1,C2,C3,C4,C5 client
class T1,T2 transport
class M1,M2,M3 middleware
class CH,FT,MD storage
class PM,LG,HC obs
class FS,WD,PA,CK,EM,DD,SW ingestReading the diagram (top → bottom):
Any MCP client — Claude Code, Cursor, Windsurf, and 5 others — connects via the transport of your choice (stdio for personal use, SSE/streamable-http for teams).
HTTP mode chains 3 ASGI middlewares in order: health probes first (always answered), then bearer auth (fenced with WWW-Authenticate), then rate limiter (sliding window).
All 13 MCP tools are decorated with @rate_limited + @instrument — metrics record calls and the rate limiter enforces RPM+burst when enabled. Disabled features skip that work, while the lightweight wrapper calls remain.
search_knowledge checks the query cache first; cache miss routes through the Query Router (regex classifier) to either the FTS5 fast-path (lexical) or the hybrid pipeline (BM25 + semantic + RRF + cross-encoder rerank).
Storage is 100% local: ChromaDB (WAL mode) for vectors + metadata, SQLite FTS5 (WAL + busy-timeout) for lexical fast-path, index_metadata.json for durable state.
Document ingestion: when enabled, watchdog observes documents/; format parsers create chunks, FastEmbed generates embeddings, and source metadata supports incremental updates. Full rebuilds use staging. See the operations guide for publication and recovery limits.
Enterprise observability (opt-in) — Prometheus /metrics, structured JSON logs, /health payload — attaches to the same instrumentation points, no code changes required.
config.yaml (with 5 domain presets) controls every subsystem — no environment variable spaghetti, no hardcoded paths.
Component ownership, query/ingestion diagrams, memory boundaries and hybrid weights: docs/ARCHITECTURE.md
Parsers use format-specific extraction: Markdown sections, code boundaries, notebook cells without base64 outputs, PDF text and spreadsheet rows. Several extensions share one parser. The runtime fallback list contains 33 formats; the documented template and presets can select a smaller list. Inspect your effective documents.supported_formats. The two MetaTrader extensions are opt-in.
# | Format | Extension | Parser | Default | Notes |
1 | Markdown |
| Section-aware (splits at | Yes | Headers preserved as chunk boundaries |
2 | Plain Text |
| Fixed-size chunking | Yes | 1000 chars + 200 overlap |
3 |
| PyMuPDF extraction or explicit page-marked UTF-8 text | Yes | No OCR; extracted-text recognition is limited to 4 MiB | |
4 | Word |
| python-docx | Yes | Headings preserved as markdown |
5 | Excel |
| openpyxl | Yes | Sheet-by-sheet extraction |
6 | PowerPoint |
| python-pptx | Yes | Slide-by-slide extraction |
7 | Jupyter Notebook |
| Cell-aware parser | Yes | Markdown + code cells only; skips outputs/base64 |
8 | JSON |
| Structure-aware | Yes | Flattened key-value extraction |
9 | CSV |
| Row-based parser | Yes | Headers + rows as text |
10 | XML |
| XML parser | Yes | Root element + namespace metadata |
11 | Python |
| Code-aware parser | Yes | Functions/classes as chunks |
12 | C Source |
| Code-aware parser | Yes | Functions / structs / includes extracted |
13 | C/C++ Header |
| Code-aware parser | Yes | Function declarations + structs extracted |
14 | C++ Source |
| Code-aware parser | Yes | Classes / structs / includes extracted |
15 | JavaScript |
| Code-aware parser | Yes | Functions / classes / imports (ESM + CJS) |
16 | React JSX |
| Code-aware parser | Yes | Same as JS parser |
17 | TypeScript |
| Code-aware parser | Yes | Functions / classes / interfaces / enums / imports |
18 | React TSX |
| Code-aware parser | Yes | Same as TS parser |
19 | Go |
| Code-aware parser | Yes | Functions / structs / imports extracted |
20 | Rust |
| Code-aware parser | Yes | Functions / structs / enums / traits / |
21 | Kotlin |
| Code-aware parser | Yes | Functions (incl. class members) / classes extracted |
22 | YAML |
| YAML parser | Yes | Kubernetes kind / apiVersion / name extracted |
23 | YAML |
| YAML parser | Yes | Same as YAML parser |
24 | HuJSON |
| HuJSON parser | Yes | JSON with comments + trailing commas (e.g. Tailscale ACLs) |
25 | CUE |
| Code-aware parser | Yes | Imports / package extracted |
26 | Protocol Buffers |
| Proto parser | Yes | Services / messages / RPCs extracted |
27 | Rego |
| Code-aware parser | Yes | OPA policies — imports / package extracted |
28 | SQL |
| SQL parser | Yes | Table names + statement types extracted |
29 | Shell |
| Shell parser | Yes | Function names extracted |
30 | jq |
| Shell parser | Yes | Indexed as shell-style script |
31 | Dockerfile |
| Text parser | Yes | Matched by exact filename (no extension) |
32 | Makefile |
| Text parser | Yes | Matched by exact filename (no extension) |
33 | Tiltfile |
| Code-aware parser | Yes | Starlark — |
34 | MQL4 Source |
| Code parser | No | MetaTrader — opt-in via |
35 | MQL4 Header |
| Code parser | No | MetaTrader — opt-in via |
Files with a .pdf extension can also contain an explicit text extraction: the
first line must be [Page N] with a positive page number, followed by nonempty
UTF-8 text. An optional UTF-8 BOM and tab/CR/LF whitespace are accepted. This
recognition is limited to 4 MiB, rejects other C0/C1 controls and DEL, and never
reinterprets a file containing PDF magic in the first 1 KiB as text. Accepted
extractions retain their source path and are marked content_format=extracted_text
and page_count_source=markers; the marker count is not proof of the original
PDF's total page count. Binary PDFs keep the native parser and its error
diagnostics. Invalid or oversized textual lookalikes remain errors.
Enable an opt-in format — add the extension to
documents.supported_formatsin yourconfig.yaml:documents: supported_formats: [".md", ".pdf", ".mq4", ".mqh"]
Full parser reference with per-format notes: docs/CONFIGURATION.md
Claude Code
~/.claude.json
Claude Desktop
claude_desktop_config.json
Cursor
~/.cursor/mcp.json
Windsurf
~/.codeium/windsurf/mcp_config.json
VS Code
Copilot Chat mcp.json
Cline · Gemini CLI · Zed Native MCP
Complete client configuration guide with JSON schemas per client: docs/INSTALLATION.md#use-with-other-mcp-clients →
⚠️ Operating model — read once. In
stdiomode, your MCP client (Claude Desktop, Code, Cursor, etc.) spawns and owns the knowledge-rag server process. Do NOT also run a secondknowledge-ragwriter in a terminal while that stdio client is open — two writer processes against the samedata_dircorrupt the ChromaDB HNSW segment on Windows (see issue #216). For HTTP transports, manually running one server and connecting HTTP clients to it is the supported shared-server workflow; do not mix it with another writer using the samedata_dir. Recommended baseline for stdio clients: add"KNOWLEDGE_RAG_SINGLE_INSTANCE": "1"to theenvblock of your MCP config as a safety net. Full explanation + how to run CLI commands safely: docs/single-instance.md.
⚠️ Do not install under cloud-sync folders. Keep
paths.data_diron a local path. OneDrive (incl.OneDrive - <tenant>), iCloud Drive, Dropbox, Google Drive, Box, pCloud, MEGAsync, Nextcloud/ownCloud and Sync all open file handles on the live index to upload it, racing with ChromaDB's HNSW mmap and SQLite WAL writes and triggering native access violations (0xC0000005on Windows,SIGBUSon *nix). Preflight auto-repair then moves the active index todata/backups/auto-repair-*and you lose the corpus. Since v4.11.0, preflight aborts withEX_CONFIG(78) whendata_dirresolves under one of these providers. Pointdata_dirat a local path (e.g.%LOCALAPPDATA%\knowledge-rag\dataon Windows,~/.local/share/knowledge-rag/dataon Linux/macOS). If you really need to run under cloud sync, setpaths.allow_cloud_sync: trueto acknowledge the risk and continue.
# config.yaml — everything is optional; defaults just work
paths:
documents_dir: "./documents"
data_dir: "./data"
models:
embedding:
profile: "compact" # "compact" | "quality" | "multilingual" | "custom"
gpu: "auto" # "auto" | "true" | "false" | "directml"
device_id: null # DirectML: explicit DXGI adapter index
reranker:
enabled: true # cross-encoder rerank
search:
default_results: 5
max_results: 100
server: # optional — SSE / HTTP mode
transport: "stdio" # or "sse" / "streamable-http"
auth:
bearer_token: "" # set a secret to enable auth
rate_limit:
enabled: false
metrics:
enabled: false
logging:
format: "text" # or "json"Pre-built presets: cybersecurity.yaml · developer.yaml · research.yaml · general.yaml · multilingual.yaml
Complete configuration reference — every field, every default, tuning guide: docs/CONFIGURATION.md →
knowledge-rag is designed for teams that cannot let their documents leave the perimeter.
Requirement | How knowledge-rag delivers |
Data locality (LGPD / GDPR / HIPAA) | 100% on-premise, zero egress network calls after initial model download |
Air-gapped deployment | ONNX models pre-cached; set |
CVE monitoring | Dependabot (weekly) + pip-audit + Socket + CodeQL |
Supply chain security | PyPI Trusted Publishing via OIDC (no long-lived tokens) |
Vulnerability disclosure | Private security advisory via SECURITY.md |
Signed release attestations | GitHub release attestations on every published version |
Dependency management | Version ranges in |
Authenticated access | Bearer token middleware on SSE / HTTP transports (constant-time compare, RFC 6750) |
Rate limiting | Sliding-window per-client RPM + burst (opt-in, zero-cost when disabled) |
Audit-ready logging | Opt-in structured JSON logs → ship to your SIEM |
Path traversal defenses | CWE-22 / CWE-59 guards on 6 CRUD tools |
External-content mitigation | Known-token neutralization and provenance fencing on |
OpenSSF Best Practices badge: passing · project ID #13864
13 MCP tools, configurable format selection and five domain presets.
Nine OS/Python CI combinations configured for Linux, macOS and Windows with Python 3.11–3.13.
Measured CPU/CUDA indexing and retrieval, AMD validation and memory regressions: audit results and reproduction.
Historical implementation benchmarks: v4.7.1 baseline. Historical ratios apply to their recorded workloads, not every corpus or competing product.
Doc | What's inside |
5 install methods · 8 MCP client integrations · GPU setup | |
Complete reference for all 13 MCP tools | |
Every | |
Components · query/ingestion flow · memory/concurrency boundaries · hybrid weights | |
11 common issues + solutions | |
Opt-in lexical fast-path — when and how | |
Zero-downtime rebuild · resume · checkpoint | |
CUDA runtime compatibility · CPU fallback · AMD/macOS GPU limitations | |
Embedding profile · multilingual · zero-downtime | |
Threat model · disclosure channel | |
Development · testing · PR process | |
All release notes since v1.0.0 | |
Reproducible workloads · measured results · validation limits · open PR/issue review |
Report a bug → Open an issue
Ask a question → GitHub Discussions
Report a vulnerability → Security advisory (private)
Contribute → CONTRIBUTING.md
Response SLA (best-effort, community project):
Security reports: within 48 h
Bug reports with reproduction: within 5 business days
Feature requests: triaged on next release cycle
v4.8.5 (2026-08-13) — Enterprise observability: /health endpoint + opt-in JSON structured logging
v4.8.4 (2026-08-13) — Patch: security + durability + defensive fixes
v4.8.3 (2026-08-10) — Critical hotfix: nuclear-rebuild + smart-reindex hardening on 50k+ chunk corpora
v4.8.2 (2026-08-10) — FTS5 lexical fast-path opt-in release
v4.8.0 (2026-08-06) — Multilingual foundation + zero-downtime reindex
Full history: CHANGELOG.md →
MIT License — LICENSE. Forever. No cloud upsell, no dual-licensing, no restrictive clauses. Fork it, sell derivatives, embed it in commercial products — the license does not care.
Built on the shoulders of amazing open-source projects:
Anthropic MCP — Model Context Protocol spec + Python SDK
ChromaDB — vector database that just works
FastEmbed — ONNX embeddings, no PyTorch bloat
HuggingFace — model hosting + Xenova/ms-marco-MiniLM-L-6-v2 cross-encoder
BAAI — the bge-small-en-v1.5 embedding model
Community contributors: @Hohlas · @eeshsaxena · Sergey Khokhlov · and everyone who filed issues or PRs.
Built by Ailton Rocha (Lyon.) · Star ⭐ if this saves you time · Report an issue · Contribute
knowledge-rag — the MCP-first local RAG server for Claude Code, Cursor, Windsurf, and every AI agent.
Add a new document to the knowledge base from raw text content.
Mutating — writes a file to disk and indexes it immediately. No auth required.
Args: content: Full text content of the document (markdown supported) filepath: Relative path within documents directory (e.g., "security/new-technique.md"). The subdirectory should match the category. category: Document category — one of: security, ctf, logscale, development, general, redteam, blueteam (default: general)
Returns: JSON string with indexing results (filepath, chunks created, status).
Usage: Use to add new documents from text content. Use add_from_url() instead when the source is a web page. Use update_document() to replace content of an existing file. The document is immediately searchable after this call — no manual reindex needed.
| Name | Required | Description | Default |
|---|---|---|---|
| content | Yes | ||
| category | No | general | |
| filepath | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavioral traits: mutating, writes to disk, indexes immediately, no auth, and returns JSON with specified fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise and well-structured: purpose statement, behavioral note, parameter descriptions, return value, and usage guidelines. Every sentence adds value without verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and 0% schema descriptions, the description covers all necessary aspects: purpose, parameters, behavior, return format, and usage context with sibling differentiation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description adds detailed semantics for all three parameters: markdown support for content, path conventions for filepath, and category enumeration with default.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool adds a document from raw text content, distinguishes from siblings like add_from_url and update_document, and mentions immediate searchability, making purpose very clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly provides when to use this tool (text content) and when to use alternatives (add_from_url for web pages, update_document for replacing files), plus notes no auth required.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fetch content from a URL, convert to markdown, and add to the knowledge base.
Mutating — makes an outbound HTTP request (requires internet access), strips HTML, converts to markdown, saves to disk, and indexes immediately.
Args: url: Full URL to fetch (https:// required). The page must be publicly accessible. category: Document category — one of: security, ctf, logscale, development, general, redteam, blueteam (default: general) title: Optional document title. Auto-detected from the page's tag if omitted.
Returns: JSON string with indexing results (detected title, filepath, chunks created, status).
Usage: Use to ingest web content (writeups, blog posts, documentation pages) directly by URL. Use add_document() instead when you already have the text content. The document is immediately searchable after this call — no manual reindex needed.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| title | No | ||
| category | No | general |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses all behavioral aspects: mutating, outbound HTTP, internet access required, HTML stripping, markdown conversion, disk save, immediate indexing. This is thorough and compensates for the lack of annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with intro, behavior, Args, Returns, and Usage sections. It is slightly verbose but efficiently conveys all necessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete: covers behavior, parameters, return value (JSON string with details), and usage context. It also mentions immediate searchability, leaving no ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description fully explains each parameter: url requires HTTPS and public accessibility, category lists possible values, title is optional and auto-detected. This adds significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool fetches content from a URL, converts to markdown, and adds to the knowledge base. It distinguishes itself from the sibling tool add_document by specifying when to use each.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit usage guidance is provided: 'Use to ingest web content... Use add_document() instead when you already have text content.' This clearly tells when and when not to use the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Evaluate search quality by testing whether search_knowledge() retrieves expected documents.
Read-only. Runs multiple search queries internally. No side effects on the index.
Args: test_cases: JSON string array of test cases. Each item requires "query" (search string) and "expected_filepath" (path of the document that should appear in top-5 results). Example: [{"query": "suid exploit", "expected_filepath": "security/suid.md"}]
Returns: JSON string with MRR@5 (Mean Reciprocal Rank), Recall@5, and per-query hit/miss breakdown. MRR@5 above 0.7 indicates good retrieval quality.
Usage: Use to audit search quality after bulk document ingestion or after tuning hybrid_alpha. Use get_index_stats() for system health checks instead. Use search_knowledge() for actual document retrieval — this tool is for quality measurement only.
| Name | Required | Description | Default |
|---|---|---|---|
| test_cases | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It says 'Read-only. Runs multiple search queries internally. No side effects on the index.' It also explains the return value structure (MRR@5, Recall@5, etc.) and expected threshold. However, it does not mention potential performance impact or rate limiting, but for a quality evaluation tool, this is reasonably transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: first sentence states purpose, then line about side effects, then parameter explanation with example, then return value description, then usage with alternatives. It is front-loaded with key info and not overly verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has only one parameter and an output schema exists (context says 'Has output schema: true'), the description is complete. It explains parameter format, return metrics, and usage context. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% because input schema only has 'test_cases' as a string with no description inside. The description provides a detailed explanation: 'JSON string array of test cases. Each item requires "query" (search string) and "expected_filepath"...' with an example. This fully compensates for the lack of schema description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Evaluate search quality by testing whether search_knowledge() retrieves expected documents.' It specifies the verb 'evaluate' and the resource 'search quality/retrieval'. It distinguishes from siblings like 'search_knowledge' and 'get_index_stats' by mentioning them in the usage section.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: 'to audit search quality after bulk document ingestion or after tuning hybrid_alpha.' It also provides alternatives: 'Use get_index_stats() for system health checks instead. Use search_knowledge() for actual document retrieval — this tool is for quality measurement only.' This is excellent guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get the full content of a specific document by filepath.
Read-only. No side effects.
Args: filepath: Relative path to the document within the documents directory (e.g., "security/technique.md"). Must be an indexed file — use list_documents() to browse available paths, or search_knowledge() to find the filepath by topic first.
Returns: JSON string with full document content and metadata (filepath, category, size).
Usage: Use when you need the complete text of a known file — search_knowledge() returns chunks, not full docs. Use search_knowledge() first to find the filepath if unknown. Use list_documents() to browse all available files by category.
| Name | Required | Description | Default |
|---|---|---|---|
| filepath | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Declares 'Read-only. No side effects.' and describes return value format. With no annotations, the description carries full burden; it could mention error behavior for missing files, but overall adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured: purpose, behavior, args, returns, usage. Each sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple tool and presence of output schema, the description covers all needed aspects: usage, parameter, return value, and context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has no description for filepath (0% coverage), but the description adds significant meaning: example path, relative path constraint, and pointers to find valid paths.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Get the full content of a specific document by filepath.' It distinguishes from siblings by noting that search_knowledge returns chunks and list_documents browses files.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use ('when you need the complete text of a known file') and provides clear guidance on alternatives (search_knowledge first if unknown, list_documents to browse).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get statistics and health metrics for the knowledge base index.
Read-only. No side effects.
Returns: JSON string with system metrics: total documents, total chunks, embedding model name, BM25 status, query cache hit rate, and file watcher status.
Usage: Use for system health checks — verifying the embedding model loaded, checking index population, or monitoring cache efficiency. Use list_categories() for per-category document counts instead. Use evaluate_retrieval() to measure actual search quality with test queries.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Declares read-only and no side effects. Lists return fields in detail: total documents, chunks, embedding model name, BM25 status, cache hit rate, file watcher status. With no annotations provided, the description fully carries the burden of transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Very concise and well-structured. Opens with purpose, then read-only note, then lists return values, then usage guidance with alternatives. Every sentence is informative with no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and an output schema, the description explains what the tool returns and its safe, read-only nature. It is completely adequate for an AI agent to understand and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so schema coverage is 100%. The description adds no parameter info because none are needed. Baseline of 4 is appropriate for zero parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool gets statistics and health metrics for the knowledge base index. It uses specific verb+resource and distinguishes from sibling tools like list_categories and evaluate_retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit usage scenarios: system health checks, verifying embedding model, checking index population, monitoring cache efficiency. Also tells when not to use by naming alternatives (list_categories for per-category counts, evaluate_retrieval for search quality).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get the current status of a background reindex operation.
Lightweight — does not compute full index statistics. Use this to poll progress after calling reindex_documents().
Returns: JSON string with reindex status. When active: operation name, progress (processed/total), percent complete, indexed/skipped/errors counts, and start time. When inactive: active=false, plus last_result or last_error from the most recent completed reindex.
Usage: Call repeatedly after reindex_documents() to monitor progress. When reindex.active becomes false, the operation is complete. Use get_index_stats() for full index health metrics.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite no annotations, the description discloses that the tool is lightweight and does not compute full index statistics. It details the return structure for both active and inactive states, providing full behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured. The first sentence gives the core purpose, followed by important details in a logical order. Returns are listed compactly, and usage instructions are clear. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and an output schema, the description thoroughly explains the return value and usage context. It also references sibling tools for completeness. The tool is simple, and the description covers all needed information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, so the input schema fully covers parameter semantics. The description adds value by explaining the return format, which is beyond what the schema provides. Baseline 4 for no parameters is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool gets the current status of a background reindex operation. It uses a specific verb ('Get') and resource ('reindex status'), and distinguishes from sibling tools like 'get_index_stats' and 'reindex_documents'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use: poll after calling reindex_documents. It also provides an alternative: use get_index_stats for full index health. This gives clear guidance on tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all document categories with their document counts.
Read-only. No side effects. Reflects the live index state.
Returns: JSON string with category names, document counts per category, and total document count.
Usage: Use before filtering search_knowledge() or list_documents() by category to see which categories exist and how many documents each contains. Use get_index_stats() instead for broader system health metrics (model name, cache hit rate, BM25 status).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description states the tool is read-only with no side effects and reflects the live index state. Since no annotations are provided, this fully covers behavioral traits such as safety and data freshness.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three short sentences, each adding value: purpose, behavior, and usage guidance. It is efficiently front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is parameterless and has an output schema, the description covers everything needed: what it does, what it returns (category names, counts, total), and when to use it. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters and schema description coverage is 100% (trivially). There are no parameters to explain, so the description adds no parameter semantics, but this is acceptable for a parameterless tool. Baseline 4 for 0 params.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all document categories with their document counts, using a specific verb ('List') and resource ('document categories'). It is distinct from siblings like 'list_documents' which lists documents, not categories.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises using this tool before filtering search_knowledge or list_documents by category, and suggests get_index_stats as an alternative for broader system health metrics. This provides clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all indexed documents, optionally filtered by category.
Read-only. No side effects.
Args: category: Optional category filter. Must be a valid category name — call list_categories() to see available options (e.g., security, ctf, logscale, development, general, redteam, blueteam).
Returns: JSON string with list of document filepaths, categories, and metadata for each indexed file.
Usage: Use to browse what's in the index or verify a specific file is indexed. Use list_categories() first to see valid category names. Use search_knowledge() when you want to find documents by topic rather than browsing the full list. Use get_document() to read a specific file once you have its filepath.
| Name | Required | Description | Default |
|---|---|---|---|
| category | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
States 'Read-only. No side effects' which is important for behavior. Discloses return format and parameter constraints. Lacks info on error handling or rate limits, but sufficient for this simple tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with summary, behavior, params, returns, and usage. Front-loaded with purpose. Slightly verbose but each sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers all needed aspects: purpose, parameters, return format, usage context, and alternatives. Output schema exists, so return details are sufficient. Annotations missing but description fills gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter 'category' has no schema description (0% coverage). The description compensates fully by explaining it's optional, must be a valid category name, and suggests calling list_categories for options, with examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (list) and resource (indexed documents) with an optional category filter. It distinguishes from siblings by mentioning when to use alternatives like search_knowledge and get_document.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance on when to use this tool (browsing index, verifying file) and when not (use list_categories first, use search_knowledge for topics, use get_document for reading). Includes specific alternative tool names.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Index or reindex all documents in the knowledge base (runs in background).
force — smart reindex (detect changed files + rebuild BM25). Use after
filesystem edits outside add_document/update_document.
full_rebuild — nuclear rebuild (delete + re-embed). Use only after
embedding-model change or index corruption. Mutually exclusive with resume.
resume — pick up an interrupted smart reindex from
data/reindex_checkpoint.json. Falls back to a fresh smart run silently
if the checkpoint is missing/corrupt/drifted (v4.8.0 Fase 4).
Returns a JSON envelope. Poll get_reindex_status() until
reindex.active becomes false. Add/update/URL tools already auto-index —
use these flags only for the recovery/rebuild scenarios above.
| Name | Required | Description | Default |
|---|---|---|---|
| force | No | ||
| resume | No | ||
| full_rebuild | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully bears the transparency burden. It discloses background execution, status polling via get_reindex_status(), silent fallback on checkpoint issues, and destructive deletion in full_rebuild ("nuclear rebuild (delete + re-embed)"). This goes well beyond basic expectations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently structured: the purpose is front-loaded, each parameter is explained on its own line, and the closing note ties everything together. Every sentence provides necessary context—no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete for a tool with three nuanced flags: it covers purpose, usage scenarios, background behavior, status polling, and the relationship to sibling tools. The output schema exists, but the description additionally clarifies the polling mechanism, making it self-sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no descriptions (0% coverage), but the description fully explains each parameter: force as smart reindex, full_rebuild as destructive rebuild with mutual exclusivity with resume, and resume with checkpoint path and fallback behavior. This adds essential meaning beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with "Index or reindex all documents in the knowledge base (runs in background)", which clearly states the verb, resource, and scope. It distinguishes itself from siblings by explicitly noting that add/update/URL tools already auto-index, so this tool is for recovery/rebuild scenarios.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit usage guidelines are provided for each flag: "Use after filesystem edits outside add_document/update_document" for force, "Use only after embedding-model change or index corruption" for full_rebuild, and a closing note that add/update/URL tools auto-index, so these flags are only for recovery/rebuild. This clearly states when to use the tool versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Remove a document from the knowledge base index.
Mutating — removes index entries. If delete_file=True, also permanently deletes the file from disk (irreversible, cannot be undone).
Args: filepath: Path to the document file. Must be an indexed document — use list_documents() to find valid paths. delete_file: If True, permanently deletes the file from disk in addition to removing from the index (default: False).
Returns: JSON string with removal results (filepath, status).
Usage: Use to unindex a document while keeping the file on disk (default). Set delete_file=True only for permanent removal. Use update_document() to replace content instead of removing. Use reindex_documents(force=True) if you deleted the file manually on disk outside of this tool.
| Name | Required | Description | Default |
|---|---|---|---|
| filepath | Yes | ||
| delete_file | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses mutating nature, removal of index entries, and irreversible deletion with delete_file=True. Since no annotations exist, description fully covers behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with summary, args, returns, usage. Every sentence adds value; no redundancy. Front-loaded with core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 2 parameters, no annotations, but with output schema referenced, description thoroughly covers behavior, return format, and alternatives. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds crucial meaning beyond schema: filepath must be an indexed document and hints to use list_documents(); delete_file explains permanence and default. With 0% schema coverage, description compensates fully.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Remove' and the resource 'document from the knowledge base index'. It distinguishes from siblings like update_document and reindex_documents, providing specific context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly guides when to use (unindex a document) and when to set delete_file=True. Provides alternatives: update_document for replacement, reindex_documents for manual deletions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Hybrid search combining semantic search + BM25 keyword search with cross-encoder reranking.
Read-only. No side effects.
Args:
query: Search query text (1–3 keywords recommended; phrase queries also work)
max_results: Maximum number of results (default: 5, max: 20)
category: Optional category filter — one of: security, ctf, logscale, development, general,
redteam, blueteam. Call list_categories() first to see available categories and counts.
hybrid_alpha: Balance between semantic and keyword search. 0.0 = keyword-only (best for exact
technical terms like CVE IDs or tool names), 0.3 = balanced default, 1.0 = semantic-only
(best for conceptual or natural-language queries).
min_score: Minimum normalized relevance score (0.0–1.0) to include a result. Results scoring
below this threshold are discarded. Default 0.0 returns all results. Use 0.2–0.4 to cut
low-relevance noise.
snippet_mode: When true (default), truncates content to ~500 characters at a natural break
point and adds a content_length field with the original size. Use get_document() to
fetch full content when needed. Set to false to return full chunk content.
search_method: Dispatch selector (v4.8.2+). One of "auto" (router picks FTS5 fast-path
for lexical queries when enabled, hybrid otherwise), "hybrid" (force hybrid path —
kill switch for suspected router misclassification), or "fts5" (force FTS5 fast-path
— debug/testing; errors out when the feature is disabled or the index is not ready).
Default "auto" preserves pre-v4.8.2 behavior byte-for-byte when the fast-path is
disabled in config.
Returns: JSON string with results including content chunks, source filepath, relevance score, and search method used. Returns chunks, not full document content.
Usage: Primary search tool — use for any topic or keyword lookup. Prefer search_similar() when you already have a reference document and want more like it. Prefer get_document() when you already know the exact filepath and need the full content.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | ||
| category | No | ||
| min_score | No | ||
| max_results | No | ||
| hybrid_alpha | No | ||
| snippet_mode | No | ||
| search_method | No | auto |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden and does so thoroughly. It discloses read-only nature ('Read-only. No side effects.'), snippet truncation behavior, min_score filtering, search_method dispatch behavior including compatibility notes, and return format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
While long, the description is well-structured with sections (overview, read-only, Args, Returns, Usage) and each sentence earns its place. It is front-loaded with the primary purpose, and the parameter details are necessary for correct invocation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the return format (JSON string with content chunks, source filepath, relevance score, search method), parameter semantics, alternatives, and edge cases like FTS5 fast-path errors. Given the tool has 7 parameters and an output schema, this description is complete without relying on external documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate, and it excels. Every parameter is explained with practical guidance: query recommendations (1-3 keywords), hybrid_alpha semantics (keyword vs semantic), min_score tuning (0.2-0.4 to cut noise), and search_method behavior including a kill switch and debug use case.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Hybrid search combining semantic search + BM25 keyword search with cross-encoder reranking,' clearly stating the tool's specific action (search) and resource (knowledge). It also differentiates from siblings by noting it 'Returns chunks, not full document content' and referencing search_similar and get_document.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The Usage section explicitly states this is the 'Primary search tool' and names direct alternatives: 'Prefer search_similar() when you already have a reference document...' and 'Prefer get_document() when you already know the exact filepath...' This provides explicit when-to-use vs when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Find documents semantically similar to a given reference document.
Read-only. No side effects. Uses the document's embedding for similarity comparison.
Args: filepath: Path to the reference document (must already be indexed — use list_documents() to verify). E.g., "security/technique.md" max_results: Number of similar documents to return (default: 5, max: 20)
Returns: JSON string with list of similar document filepaths and similarity scores (0.0–1.0).
Usage: Use when you have a specific document and want to discover thematically related ones. Use search_knowledge() instead when you have a text query rather than a reference document. The reference document must be indexed — call list_documents() to confirm it exists before calling this tool.
| Name | Required | Description | Default |
|---|---|---|---|
| filepath | Yes | ||
| max_results | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description fully carries the burden. It states 'Read-only. No side effects.' and explains the use of embeddings for similarity. Return format is described in detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is concise and well-structured: brief summary, behavioral note, args list, returns, usage. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and presence of output schema, description covers all needed aspects: purpose, usage, parameters, behavior, and return format. No gaps identified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description compensates by fully explaining both parameters: filepath (path, must be indexed, example) and max_results (default and max). This adds significant meaning beyond the bare schema types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states action: 'Find documents semantically similar to a given reference document.' It distinguishes from sibling tool 'search_knowledge' by specifying it works with a reference document, not a text query.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: when you have a specific document, and when not: use search_knowledge for text queries. Also includes prerequisite that document must be indexed and suggests verifying with list_documents().
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Update the content of an existing document in the knowledge base.
Mutating — overwrites the file on disk and re-indexes immediately. Old chunks are removed and replaced with new ones. Full content replacement, not a patch.
Args: filepath: Full or relative path to the document file. Must be an already-indexed file — use list_documents() to find valid paths. content: New full-text content to replace the existing content entirely
Returns: JSON string with update results (old chunk count, new chunk count, status).
Usage: Use to replace a document's content completely. Use add_document() to create a new file instead. Use remove_document() to delete without replacing. Changes are immediately searchable — no manual reindex needed.
| Name | Required | Description | Default |
|---|---|---|---|
| content | Yes | ||
| filepath | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description fully discloses behavior: 'Mutating — overwrites the file on disk and re-indexes immediately. Old chunks are removed and replaced with new ones. Full content replacement, not a patch.' Also states changes are immediately searchable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with separate sections for behavior, args, returns, and usage. Every sentence is informative, though slightly verbose. Front-loaded with purpose and behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with reindexing and full replacement, description covers prerequisites, effects, and output. References sibling tools for alternative actions. No gaps given the complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description explains both parameters: filepath must be an already-indexed file and reference to list_documents(), content is full-text replacement. Also describes return value format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Update the content of an existing document' with specific verb and resource. Distinguishes from siblings add_document and remove_document directly in the usage section.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when to use (replace content completely), when not (use add_document for new files, remove_document for deletion), and prerequisite (file must already be indexed, use list_documents to find paths).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v4.8.3search_knowledge1 field changedInput schema / properties / search_methodAdded value: +{
+ "default": "auto",
+ "title": "Search Method",
+ "type": "string"
+}v4.8.1reindex_documents1 field changedInput schema / properties / resumeAdded value: +{
+ "default": false,
+ "title": "Resume",
+ "type": "boolean"
+}v4.3.1get_reindex_statussearch_knowledge2 fields changedInput schema / properties / min_scoreAdded value: +{
+ "default": 0,
+ "title": "Min Score",
+ "type": "number"
+}Input schema / properties / snippet_modeAdded value: +{
+ "default": true,
+ "title": "Snippet Mode",
+ "type": "boolean"
+}v3.0.0add_documentadd_from_urlevaluate_retrievalget_documentget_index_statslist_categorieslist_documentsreindex_documentsremove_documentsearch_knowledgesearch_similarupdate_documentScored across 13 tools
Each tool has a clearly distinct purpose: search, fetch full document, list, CRUD operations, URL import, and index maintenance. The two status/stat tools (get_reindex_status, get_index_stats) are separate and non-overlapping. No ambiguity between search and retrieval, or between add and update.
Most tools follow a consistent verb_noun pattern (search_knowledge, add_document, update_document, remove_document, list_categories, list_documents). Minor deviation with 'add_from_url' which mixes source into the name, and 'evaluate_retrieval' which is a compound but still understandable. Overall the naming is predictable and clear.
13 tools is a reasonable number for a knowledge base management system, covering search, document CRUD, URL import, reindexing, and evaluation. This is within the ideal 3-15 range and not excessive or sparse.
The tool set covers the full lifecycle: adding (via text or URL), updating, deleting, searching, retrieving, listing, reindexing, and quality evaluation. There's no obvious missing functionality for a RAG knowledge base. The inclusion of evaluate_retrieval and index stats adds operational completeness.
Cloud or self-hosted knowledge for AI agents: hybrid search, reranking, GraphRAG, scoped MCP tools.
Persistent, outcome-grounded episodic memory for Claude. 14ms CPU retrieval, no GPU, no vector DB.
Private persistent memory for Claude, ChatGPT & Gemini via MCP - semantic search, zero-code setup.
No-data MCP handoff for local Claude Code to Codex harness moves. $49 lifetime.
