# notesearch-rag

Hybrid search over a folder of markdown notes (for example Claude Code's auto-memory), as a command
line tool and as an MCP server for Claude Code, LM Studio and any other MCP client.

It follows the "raw-history RAG" idea from [Is Complex Agentic Memory Always Needed? Naive RAG Is
Stronger Than We Thought](https://arxiv.org/abs/2610.30755) (code: [Litmeb/RAW-RAG](https://github.com/Litmeb/RAW-RAG)):
keep the original text, retrieve relevant windows, let the reader model do the reasoning. This is an independent,
much smaller implementation and shares no code with that repository.

* Notes are cut into overlapping raw-text windows (about 1500 characters), each prefixed with the note's name and description.
  Nothing is summarised.
* Retrieval fuses dense vectors (any OpenAI-compatible `/v1/embeddings` server, default LM Studio with `bge-m3`) with BM25
  keyword search by reciprocal rank fusion. BM25 keeps IPs, hostnames and flags as whole tokens as well as their parts.
* The index refreshes itself: only windows whose text changed are re-embedded. If the embedding server is down it
  falls back to keyword-only search and says so.
* Runtime dependencies: Python, numpy, requests. No GPU or torch needed.

## Use

    notesearch --update                 # build/refresh the index once
    notesearch "why does the printer web page time out"
    notesearch -n 3 -v --json "dhcp reservation"

## MCP server

`notesearch-mcp` speaks MCP over stdio and provides two read-only tools: `search_notes` and `read_note`.
Build the index once with `notesearch --update` first.

Claude Code:

    claude mcp add --scope user notesearch -- notesearch-mcp

LM Studio (`~/.lmstudio/mcp.json`):

    { "mcpServers": { "notesearch": { "command": "notesearch-mcp" } } }

## Configuration (environment variables)

| Variable | Default |
|---|---|
| `NOTESEARCH_NOTES` | `~/.claude/projects/<home path with / as ->/memory` |
| `NOTESEARCH_DATA` | `$XDG_DATA_HOME/notesearch` (the index) |
| `NOTESEARCH_EMBED_URL` | `http://127.0.0.1:1234/v1/embeddings` |
| `NOTESEARCH_EMBED_MODEL` | `text-embedding-bge-m3-latest` |
| `NOTESEARCH_EMBED_KEY` | empty (optional bearer token) |
| `NOTESEARCH_SKIP` | `MEMORY.md` (comma-separated file names to leave out) |

Changing the embedding model rebuilds the index automatically.

## How well does it work?

On a private collection of about 440 notes, with 25 self-written paraphrased questions, the right note was in the top 5
for 92% of questions with hybrid search, against 40% for a grep-style search and 68% for BM25 alone. A cross-encoder
reranker added nothing at 13 s per query on CPU, so it is not included. The questions were written by the author and are
optimistic; treat this as a rough comparison, not a benchmark. Retrieved notes can be stale, so verify before acting on them.

## Tests

    PYTHONPATH=src python -m unittest discover -s tests

## License

MIT