<p align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="docs/assets/logo-dark.svg">
    <img alt="Ghostwire" src="docs/assets/logo-light.svg" width="440">
  </picture>
</p>

<p align="center">
  <strong>A local-first memory layer for AI tools.</strong><br>
  Your notes and browser history, searchable by Claude and any MCP client, without leaving your machine.
</p>

<p align="center">
  <a href="https://github.com/Scrasaa/ghostwire/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/Scrasaa/ghostwire/actions/workflows/ci.yml/badge.svg"></a>
  <img alt="Python 3.11+" src="https://img.shields.io/badge/python-3.11%2B-3776AB?logo=python&amp;logoColor=white">
  <img alt="Linux, macOS, Windows" src="https://img.shields.io/badge/platform-linux%20%7C%20macOS%20%7C%20windows-6E7781">
  <img alt="MCP server" src="https://img.shields.io/badge/MCP-server-7C5CFF">
  <a href="LICENSE"><img alt="MIT license" src="https://img.shields.io/badge/license-MIT-22C3EE"></a>
</p>

Ghostwire indexes your notes and browser history into a local SQLite database (FTS5 +
sqlite-vec) and exposes it through a local MCP server. It is open source, and everything stays
on your machine.

- **Markdown notes**: `ghostwire import markdown`, kept current with `ghostwire watch`
- **Browser history**: page titles and URLs from Firefox and Chromium-family browsers
- **Hybrid search**: keyword (FTS5/BM25) plus semantic vectors from a local Ollama
- **MCP server**: `search_memory`, `get_document`, `add_memory` and `list_sources` for Claude
  Desktop, Claude Code or any other MCP client

Status: pre-release (0.1.0). Linux, macOS and Windows, Python 3.11+.

## Why Ghostwire

AI assistants don't know what's in your notes or what you've read online. You end up
copy-pasting context into every chat, or uploading your notes to a hosted service. Ghostwire
gives the assistant a search tool over your own data, and that data never leaves your machine.

What you can use it for:

- **Ask questions about your notes.** "What did I decide about the auth migration?" Claude
  searches your notes folder (Obsidian or any other Markdown folder) and answers from what
  you wrote.
- **Find pages you visited before.** "Find the article about reciprocal rank fusion I read last
  week." Ghostwire searches the titles and URLs in your browser history, so you don't have to
  remember the exact words.
- **Keep context between chats.** The assistant can save facts, decisions or summaries with
  `add_memory` and find them again in later conversations.
- **Search it yourself.** `ghostwire search` runs the same search from the terminal, without
  an AI client.

Why use it instead of a hosted "second brain":

- **Private:** the index is one SQLite file on your disk, and embeddings come from a local
  Ollama. Your data is never sent to a cloud service.
- **Finds the right note:** keyword search matches exact names and terms, and semantic search
  finds notes that use different words for the same idea.
- **Works with any MCP client:** Claude Desktop, Claude Code or any other client that speaks
  MCP. You aren't tied to one app.
- **Little upkeep:** `ghostwire watch` updates the index as you edit your notes. Without
  Ollama, keyword search still works.

## How it works

```mermaid
flowchart TB
    notes["Markdown notes"] -->|"import / watch"| imp
    hist["Browser history<br/>titles + URLs only"] -->|"import browser"| imp

    subgraph gw["Ghostwire"]
        imp["Importers<br/>chunk · hash · skip unchanged"] --> db[("SQLite<br/>FTS5 + sqlite-vec")]
        db --> search["Hybrid search<br/>BM25 + vectors → RRF"]
        search --> cli["CLI<br/>ghostwire search"]
        search --> mcp["MCP server<br/>ghostwire serve (stdio)"]
        mcp -->|"add_memory"| db
    end

    imp -.->|"embed new chunks"| ollama["Ollama<br/>localhost only"]
    search -.->|"embed query"| ollama
    mcp <-->|"search_memory · get_document<br/>add_memory · list_sources"| clients["Claude Desktop · Claude Code<br/>any MCP client"]
```

Importers split each document into overlapping chunks and store them in a single SQLite file.
Unchanged files are skipped by content hash. When Ollama is running, new chunks also get an
embedding vector; without it, everything still works with keyword search.

### Hybrid search

```mermaid
flowchart TD
    q(["Query"]) --> fts["Keyword query<br/>drop stopwords, quote each term"]
    q --> emb["Embed query<br/>(Ollama)"]
    fts --> bm25["FTS5<br/>BM25 ranking"]
    emb --> knn["sqlite-vec<br/>cosine nearest neighbours"]
    bm25 --> rrf["Reciprocal Rank Fusion<br/>score = Σ 1 / (60 + rank)"]
    knn -->|"keep if similarity ≥ min_similarity"| rrf
    rrf --> out(["Top results"])
    bm25 -.->|"Ollama down or no vectors"| kw(["Keyword results only"])
```

RRF uses only each result's rank in the two lists, so BM25 scores and cosine distances never
need to be put on the same scale. A chunk found by both searches rises to the top.

## Install

Ghostwire is not on PyPI yet; install it from GitHub with [uv](https://docs.astral.sh/uv/):

```sh
uv tool install git+https://github.com/Scrasaa/ghostwire
```

(`pipx install git+https://github.com/Scrasaa/ghostwire` works too.)

For semantic search, install [Ollama](https://ollama.com) and pull the embedding model.
Without it everything still works, with keyword search only.

```sh
ollama pull nomic-embed-text
```

Vector search needs a Python that can load SQLite extensions. If `ghostwire init` warns that it
cannot, reinstall with uv's own Python build:
`uv tool install --python-preference only-managed git+https://github.com/Scrasaa/ghostwire`.

## Quickstart

```sh
ghostwire init                       # write config.toml and create the database
ghostwire import markdown ~/notes    # index a folder of .md files
ghostwire import browser             # optional: titles and URLs from browser history
ghostwire search "sourdough starter"
ghostwire status
```

To use it from Claude Desktop or Claude Code, see [docs/mcp-setup.md](docs/mcp-setup.md).

To try it without touching your own data, run the demo from a clone of this repository. It
indexes a few sample notes in a temporary folder, searches them, and deletes everything:

```sh
uv run python scripts/demo.py
```

## Search

```sh
ghostwire search "sourdough starter" --limit 5 --source markdown
```

Keyword (FTS5/BM25) and vector results are merged with Reciprocal Rank Fusion. Vector hits
below a cosine similarity of `min_similarity` (default 0.5, in the `[search]` section of
`config.toml`) are dropped, so a query about something you never saved mostly returns nothing
instead of the least-unrelated notes. Keyword hits are never filtered. With `nomic-embed-text`,
unrelated text still scores about 0.35–0.55 and good matches about 0.6–0.7. Lower the value if
relevant results go missing (browser pages, which are indexed by title and URL only, score
lower), or set it to 0 to keep every vector hit.

## Keeping notes up to date

```sh
ghostwire watch ~/notes ~/work/wiki   # sync, then re-import whenever a note changes
ghostwire watch                       # same, for watch_folders in config.toml
ghostwire watch --once                # sync watch_folders once and exit (for cron)
```

Each folder is synced at start, then re-imported once its files have been quiet for half a
second; only folders with changes are re-imported, and new text is embedded right away when
Ollama is running. Non-Markdown files and anything inside dot-folders (`.git`, `.obsidian`) are
ignored. If a watched folder disappears (an unmounted drive, a rename), it is skipped and its
notes stay in the index. Stop with Ctrl+C or SIGTERM. The database uses SQLite WAL mode, so the
MCP server can answer searches while `watch` writes.

To run it in the background on Linux, a systemd user service works:

```ini
# ~/.config/systemd/user/ghostwire-watch.service
[Unit]
Description=Ghostwire note watcher

[Service]
ExecStart=%h/.local/bin/ghostwire watch
Restart=on-failure

[Install]
WantedBy=default.target
```

Then `systemctl --user enable --now ghostwire-watch`.

## Browser history

```sh
ghostwire import browser --list      # history databases found (Firefox, Chrome, Chromium,
                                     # Brave, Edge, Vivaldi, LibreWolf)
ghostwire import browser             # index all of them
ghostwire import browser ~/.mozilla/firefox/abc.default/places.sqlite --days 90
```

Only page titles and URLs are indexed; pages are never downloaded. The browser's database is
copied to a temporary folder and read there, so the browser can stay open. Skipped: non-http(s)
URLs, URLs containing a username or password, and domains on `domain_blocklist` in the
`[browser]` section of `config.toml` (subdomains included; banking and webmail by default).
Tracking parameters (`utm_*`, `fbclid`, `gclid`, …) and parameters that usually hold secrets
(`token`, `access_token`, `session`, `sig`, …) are removed from URLs; other parameters such as
`?v=` or `?q=` are kept. Re-running removes pages that are now blocked or, with `--days`, older
than the window.

## Configuration

`ghostwire init` writes `config.toml` to `~/.config/ghostwire/` (on every OS). Set
`GHOSTWIRE_HOME` to use a different folder. Missing keys fall back to the defaults.

| Key | Default | Meaning |
| --- | --- | --- |
| `db_path` | `~/.config/ghostwire/ghostwire.db` | The SQLite database |
| `ollama_url` | `http://localhost:11434` | Ollama server; must be on localhost |
| `embedding_model` | `nomic-embed-text` | Changing it re-embeds everything on the next import |
| `watch_folders` | `[]` | Folders `ghostwire watch` syncs when given none |
| `[browser] blocklist_enabled` | `true` | Apply `domain_blocklist` |
| `[browser] domain_blocklist` | banking and webmail domains | Never index these (subdomains too) |
| `[search] min_similarity` | `0.5` | Drop vector hits below this cosine similarity (0–1) |

## Privacy

- Everything is stored in one SQLite file (`db_path`) and the config folder. Nothing else is
  written, apart from a temporary copy of a browser's history database during an import.
- The only network connection is to Ollama, and Ghostwire refuses an `ollama_url` that is not
  on localhost. There is no telemetry and no update check.
- Browser pages are never downloaded: only titles and URLs are stored, with tracking and
  secret-bearing query parameters removed.
- Document contents are never written to logs.
- To delete everything, remove `~/.config/ghostwire/` (or your `GHOSTWIRE_HOME`) and the
  `db_path` file if you moved it.

## Development

```sh
uv sync
uv run pytest
uv run ruff check . && uv run ruff format --check .
uv run mypy src
```

CI runs the same checks and the demo on Linux, macOS and Windows with Python 3.11 and 3.13,
and builds the package. See [CONTRIBUTING.md](CONTRIBUTING.md).