Skip to main content
Glama
README.md
# OSI — Obsidian Semantic Index

Read-only semantic search and similar notes over an Obsidian vault, mainly for LLMs over MCP.
**Assumed setup.** The vault reaches the server through a sync tool and is mounted read-only. Embeddings come
from a [llama.cpp](https://github.com/ggml-org/llama.cpp) server in router mode, with one GPU entry and one
CPU entry for the same GGUF file. The GPU is shared with other models, such as chat models.

- **search**: keyword (SQLite FTS5) and vector search, merged with reciprocal rank fusion, grouped by note,
  with the best 1–3 matching sections per note.
- **similar**: precomputed similar notes with the section pairs that explain each match and their link status.
- **get_note**: a note or one section, with frontmatter, outgoing links and backlinks.
- **status**: index health.

The same four functions are served as REST routes and as MCP tools at `/mcp`. Everything needs a bearer key
except `/health`.

## Layout

| Path | Contents |
| --- | --- |
| `src/osi/` | The service: config, markdown parsing, chunking, links, SQLite store, embedder client, search, similar notes, indexer thread, REST, MCP, CLI |
| `tests/` | pytest suite with a synthetic fixture vault and a fake embedder (no llama.cpp needed) |
| `obsidian/similar.js` | Dataview view that shows similar notes inside a note |
| `skill/SKILL.md` | Instructions for LLM harnesses on when to use which tool |
| `llama/router-preset.ini` | Embedder entries for the llama.cpp router |
| `scripts/smoke.sh` | curl checks against a running service |
## Build and test (on the server)

Nothing runs on the development machine; the server builds and tests.

```bash
docker build --target test -t osi:test .        # runs the test suite; fails the build on any failure
docker build --target runtime -t osi:0.1.0 .    # the service image; the tag must match IMAGE_TAG in .env
```

**Lock file.** The test stage records exact dependency versions. Copy them back into the repo once:

```bash
docker run --rm osi:test cat /app/requirements.lock > requirements.lock
```

## Configure and run

1. The service needs only `compose.yaml` and `.env`, so it can run from any folder on the machine that built
   the image. `cp .env.example .env` and fill it in; `IMAGE_TAG` names the image you built. Generate keys with
   `openssl rand -hex 32`. To update, build a new tag, change `IMAGE_TAG` and run `docker compose up -d`.
2. Add the two entries from `llama/router-preset.ini` to the router. `EMBED_MODEL_ID` is the GGUF file name
   without `.gguf` (`Qwen3-Embedding-8B-Q8_0`); OSI checks it against both entries. The model, the router
   entries, the nightly time and the update mode are all set in `.env`.
3. `docker compose up -d`, then check `docker compose logs -f osi`. A missing or malformed setting stops the
   service at startup with a list of every problem.
4. The keyword index builds on its own within a minute. Build the vectors once:

   ```bash
   docker compose exec osi osi reindex          # refuses while another model is loaded on the router
   docker compose exec osi osi reindex --force  # runs anyway
   docker compose exec osi osi status           # watch build.state, pending_vectors, vector_search
   ```

5. Run `scripts/smoke.sh` with a client key.

### Admin commands

All run inside the container and use `ADMIN_KEY`.

| Command | Does |
| --- | --- |
| `osi status` | Full status: counts, freshness, fingerprint, build progress, recent errors |
| `osi scan` | Scan the vault now |
| `osi embed` | Run the nightly update now: pending chunks on the GPU entry, or the CPU entry while the GPU is in use |
| `osi reindex [--force]` | Full vector build on the GPU entry (or CPU if `EMBED_GPU_URL` is empty) |
| `osi check [--force] [--samples 50]` | Embed sample chunks on CPU and GPU and compare: cosine and top-10 overlap |
| `osi eval [-k 10] [--file ...]` | recall@k and MRR for keyword, vector and hybrid search, plus similar notes |
| `osi bench [...]` | One-off model benchmark against the router; needs no `.env` or running service (see below) |

### Benchmark: choosing the model

`osi bench` reads the vault exactly as the indexer does and, for each model, measures:
- a full GPU pass over every chunk, plus the similar-notes recompute (the cost of a nightly rebuild);
- CPU query latency, CPU throughput on a sample, and how closely CPU vectors match GPU vectors;
- recall@10 for vector search and similar notes, if you pass the test set.

It prints a Markdown report with numbers only (no note paths, titles, prompts or text) and sends progress
to stderr. It refuses to start while another model is loaded on the router (`--force` overrides), stops if
one gets loaded during a GPU pass, and unloads every entry it used.

1. Make sure the router has the entries to compare. The defaults are the 4B and 8B Q8_0 pairs in
   `llama/router-preset.ini`.
2. On the server:

   ```bash
   docker build --target runtime -t osi:bench .
   docker run --rm --user "$(id -u):$(id -g)" \
     -v /path/to/vault:/vault:ro \
     -v /path/to/data/eval:/eval:ro \
     osi:bench osi bench --url http://<router>:<port> \
       --exclude-folders "Templates,Private" --exclude-tags private \
       --testset /eval/testset.yaml > bench.md
   ```

   - Without a test set, leave out the `/eval` mount and `--testset`.
   - With a `.env`, `--env-file .env` supplies the exclusions and chunk settings instead of the flags.
   - Add `--limit-chunks 300` for a quick trial run first.
3. Entries the router lacks are skipped. `--pair LABEL=GPU_ENTRY,CPU_ENTRY[,CPU_ENTRY...]` (repeatable)
   picks other entries.

The default model, Qwen3-Embedding-8B Q8_0, was chosen this way. On a vault of about 2,400 notes a full GPU
pass took about 4 minutes, CPU queries took about 0.2 s with 16 threads, and at Q8_0 CPU and GPU vectors were
interchangeable (cosine 0.999 or higher). Your numbers will differ; run `osi bench` on your own hardware.

### Test set

`osi eval` reads `/data/eval/testset.yaml` (inside the data folder, never in the repo, since it names notes):

```yaml
search:
  - prompt: "a question you would ask"
    expect: ["Folder/Expected note.md", "Other/Note.md"]
similar:
  - note: "Folder/Some note.md"
    expect: ["Folder/Related note.md"]
```

## Clients

**opencode** (`opencode.json`, current config format: servers under `mcp.servers`, `oauth` off for key auth):

```json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "servers": {
      "osi": {
        "type": "remote",
        "url": "http://<server>:<API_PORT>/mcp",
        "oauth": false,
        "headers": { "Authorization": "Bearer {env:OSI_KEY}" }
      }
    }
  }
}
```

**Claude Code**: `claude mcp add --transport http osi http://<server>:<API_PORT>/mcp --header "Authorization: Bearer $OSI_KEY"`

**Other MCP clients**: Streamable HTTP at `http://<server>:<port>/mcp` with the header
`Authorization: Bearer <key>`. Give each client its own key so it can be revoked on its own and shows up
separately in the query log.

**Obsidian**: see the header of `obsidian/similar.js`.

**Skill**: copy `skill/SKILL.md` to `~/.claude/skills/vault-notes/SKILL.md`, which both Claude Code and
opencode read. Then replace the server placeholder and fill in what your folders mean. The skill names the
tools for each harness and falls back to the REST API when no MCP tool is registered. For harnesses
without skills, paste it into the system prompt.

## How it behaves

- **Who does what**: the GPU is assumed to be shared with other work, such as chat models, and busy during
  the day. So the CPU entry only embeds search queries (a fraction of a second each), and all bulk
  embedding goes to the GPU entry at night, which is unloaded afterwards.
- **Updates**: the vault is scanned every `SCAN_SECONDS`, so keyword search, links, tags, filters and
  `get_note` follow edits within a minute. Edited sections are found by meaning after the nightly run, which
  starts at `NIGHTLY_TIME`, embeds only chunks without a vector, then recomputes the similar lists. If
  another model is loaded on the router, the run waits; in the window's last `NIGHTLY_CPU_FALLBACK_MINUTES`
  the CPU entry takes over. `UPDATE_MODE=debounce` adds daytime embedding on the CPU,
  `DEBOUNCE_MINUTES` after a note's last change. `/status` shows `last_embed` and `update_waiting`.
- **Guards**: vector search switches off, with the reason in `/status` and in each search response, when the
  embedding settings changed since the last build, when the CPU embedder serves a different GGUF than
  `EMBED_MODEL_ID`, or when its probe vectors drift. Search then falls back to keywords until `osi reindex`.
- **Sharing the GPU**: a full build refuses to start while another model is loaded on the router (unless
  forced), steps aside if one gets loaded during the build, and unloads the GPU embedder at the end.
- **Privacy**: excluded folders, hidden folders and notes with an excluded tag are never stored; get_note
  answers 404 for them exactly as for missing notes. `%%comments%%`, HTML comments and Templater tags are
  stripped from the index and from get_note output. The query log stays in `/data` and is purged after
  `QUERY_LOG_RETENTION_DAYS`.

## License

MIT; see `LICENSE`.