# MCP server — Obsidian vault access

Remote MCP server (Streamable HTTP transport, OAuth 2.1 + PKCE auth) that
gives Claude read/search/archive access to a folder of markdown notes (an
Obsidian vault).

Auth is OAuth rather than a plain static token because claude.ai's custom
connector UI only has fields for a URL and an OAuth Client ID/Secret — there
is no field for a raw bearer token. See "Notes on auth" below for what that
means in practice.

## Project layout

```
app/
  config.py          # .env-driven settings (VAULT_PATH, OAUTH_CLIENT_ID/SECRET, HOST, PORT, LOG_LEVEL, LOG_FILE, PUBLIC_HOSTNAME)
  mcp_instance.py     # the shared FastMCP instance, wired with the OAuth provider + DNS-rebinding protection
  oauth_provider.py   # minimal single-tenant OAuth 2.1 authorization server (see below)
  audit_log.py        # file-based audit logger, wraps every tool call
  vault.py            # path-safety + filesystem logic (list/search/read/edit/archive)
  tools.py            # the MCP tools, thin wrappers over vault.py
  server.py           # builds the ASGI app from the FastMCP instance
main.py               # entrypoint: uvicorn.run(app, host=..., port=...)
sample_vault/         # tiny fixture vault for local testing
scripts/manual_test.py  # scripted client: runs the OAuth dance, then exercises all tools
deploy/mcp-obsidian.service  # example systemd (--user) unit
```

## Tools exposed

1. `read_index()` — reads the vault's entry point, configured via `INDEX_PATH` in `.env` (default `README.md`). Call this first.
2. `list_notes(folder=None, recursive=False)` — lists files/folders, excludes `_trash/`. `recursive=True` walks the whole subtree in one call.
3. `search_notes(query, limit=10)` — full-text search across `.md` files, excludes `_trash/`.
4. `read_note(path)` — returns full file content.
5. `create_note(path, content)` — creates a new file; fails if one already exists there.
6. `edit_note(path, content)` — overwrites an existing file's full content; fails if it doesn't exist yet.
7. `str_replace_note(path, old_str, new_str)` — replaces one exact, uniquely-matching occurrence of `old_str` without resending the whole file.
8. `append_note(path, text)` — appends text to the end of an existing file.
9. `insert_in_note(path, anchor, text, position="before")` — inserts a line before/after the line containing `anchor`.
10. `archive_note(path)` — moves a note into `_trash/` (never deletes physically).
11. `get_unread_files(limit=5)` — pulls up to `limit` not-yet-read files out of `raw/_status.md` (see "Syncing Yandex.Disk into raw/" below), extracts real content from PDF/DOCX/photos (see below), and marks them read as part of the same call.

`create_note`/edit-family tools can't write directly into `_trash/` — that
tree is only ever populated by `archive_note`.

Prefer `str_replace_note`/`append_note`/`insert_in_note` over `edit_note` for
partial changes — they send only the changed text instead of the whole file,
which matters a lot once notes grow past a hundred lines or so. Every write
tool returns a `last_modified` timestamp confirming the change landed, so
there's no need to follow up with a defensive `read_note`.

All path-taking tools reject absolute paths and `..` segments (path traversal
protection lives in `app/vault.py::resolve_safe_path`).

### A note on eventual consistency

`VAULT_PATH` is expected to be kept in sync with the user's real vault by
something outside this process (e.g. a sync tool watching the same
directory). That means a file can very briefly disappear and reappear from
under us — in practice this can surface as `list_notes` returning an
incomplete listing right after other changes landed, or a write tool
failing with "not a file" for a path that demonstrably exists a moment
later. All write tools (`edit_note`, `str_replace_note`, `append_note`,
`insert_in_note`, `archive_note`) retry the existence check briefly (up to
~0.2s) before failing, to absorb that window. `list_notes` has no
equivalent check to retry against, so its tool description tells the
calling agent to verify with `search_notes`/`read_note` before concluding a
note is missing, rather than trusting a single listing.

## Local setup

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
```

Edit `.env`:
- `VAULT_PATH=./sample_vault` for local testing (a real vault copy also works).
- `OAUTH_CLIENT_ID` / `OAUTH_CLIENT_SECRET` — set real random values, e.g. `openssl rand -hex 32` for each.
- Leave `PUBLIC_HOSTNAME` empty for local-only use.

Run it:

```bash
python main.py
```

It binds to `127.0.0.1:8000` by default (see `.env`). The MCP endpoint is at
`http://127.0.0.1:8000/mcp`; OAuth endpoints (`/authorize`, `/token`,
`/.well-known/oauth-authorization-server`, …) live at the same host:port.

## Testing before connecting to claude.ai

**1. Auth check with curl** — hitting the MCP endpoint with no token must
return `401` with a `WWW-Authenticate` header pointing at the protected
resource metadata (this is what tells claude.ai where to find the OAuth
endpoints):

```bash
curl -i http://127.0.0.1:8000/mcp
curl -i http://127.0.0.1:8000/.well-known/oauth-authorization-server
```

**2. Scripted smoke test** — runs the full OAuth 2.1 + PKCE flow against
your own server (no browser needed — `app/oauth_provider.py` auto-approves,
since the real gate is knowing the client secret), then exercises all 5
tools against `sample_vault/`:

```bash
OAUTH_CLIENT_ID=<from .env> OAUTH_CLIENT_SECRET=<from .env> python scripts/manual_test.py
```

The script creates, edits, and archives a scratch file under
`sample_vault/_scratch/` as part of the run, so it's safe to re-run
repeatedly without resetting `sample_vault/`.

**3. MCP Inspector** (interactive, closest to how claude.ai will talk to it):

```bash
npx @modelcontextprotocol/inspector
```

In the UI: Transport = `Streamable HTTP`, URL = `http://127.0.0.1:8000/mcp`.
Inspector will detect the `401` + metadata and walk you through the OAuth
flow itself, prompting for the Client ID/Secret from your `.env`.

## Connecting to claude.ai

Team/Enterprise plan, as an **owner**: Admin settings → Connectors → Add
custom connector →

- **URL:** `https://<your-public-hostname>/mcp`
- **Advanced settings → OAuth Client ID:** value of `OAUTH_CLIENT_ID`
- **Advanced settings → OAuth Client Secret:** value of `OAUTH_CLIENT_SECRET`

Each **member** then goes to Settings → Connectors, finds the connector, and
clicks "Connect" — this runs them through the OAuth consent screen (which
auto-approves) and gets them their own access token.

## Deploying on the VPS (systemd, user-level service)

This runs as a `systemctl --user` service under your own account — no
dedicated system user or root-owned `/opt` directory needed. The only root
actions required, ever, are creating `/vault` (owned by your user) and
enabling "lingering" so the user service can run without an active login
session.

1. One-time, as root (or via `sudo`):

   ```bash
   sudo mkdir -p /vault && sudo chown "$USER":"$USER" /vault
   sudo loginctl enable-linger "$USER"
   ```

2. Copy the project to `~/mcp-obsidian` on the VPS, create a venv there,
   `pip install -r requirements.txt`.
3. Create `~/mcp-obsidian/.env` with `VAULT_PATH=/vault`, strong
   `OAUTH_CLIENT_ID`/`OAUTH_CLIENT_SECRET` values, `PUBLIC_HOSTNAME` set to
   the hostname your reverse proxy serves (e.g. a nip.io address or your own
   domain), and `LOG_FILE=~/mcp-obsidian/logs/server.log` (expand `~` to the
   real home path — systemd `EnvironmentFile` doesn't expand `~`).
4. Install the unit file:

   ```bash
   mkdir -p ~/.config/systemd/user
   cp deploy/mcp-obsidian.service ~/.config/systemd/user/
   systemctl --user daemon-reload
   systemctl --user enable --now mcp-obsidian
   systemctl --user status mcp-obsidian
   ```

5. Point your reverse proxy (Caddy/nginx) at `127.0.0.1:8000`, forwarding
   the `Authorization` header through unchanged (this is the default
   behavior for both — just don't strip it in your config). The proxy also
   needs to own HTTPS for the exact hostname in `PUBLIC_HOSTNAME`, since
   that hostname is baked into the OAuth issuer/resource URLs.

## Auto-deploy (GitHub Actions)

Every push to `main` runs `.github/workflows/deploy.yml`, which:

1. Rsyncs the repo to `/home/durov/mcp-obsidian` on the VPS (never touching
   `.env`, `logs/`, or `.venv/` there — those are excluded).
2. Runs `pip install -r requirements.txt` in the existing venv.
3. Restarts the `mcp-obsidian` user service.
4. Hits `/mcp` locally on the VPS and fails the workflow if it doesn't get
   the expected `401` (i.e. the service didn't come back up healthy).

This needs exactly one GitHub Actions secret, since the VPS host/user and
its SSH host key are already pinned in the workflow file:

- **`VPS_SSH_PRIVATE_KEY`** — private key of a dedicated deploy keypair
  (its public half is already installed in `~/.ssh/authorized_keys` for
  `durov` on the VPS, separate from any personal/interactive SSH key).

Add it at: repo → **Settings → Secrets and variables → Actions → New
repository secret**.

You can also trigger a deploy manually from the **Actions** tab (workflow
has `workflow_dispatch` enabled) without pushing a commit.

## Syncing Yandex.Disk into raw/

`scripts/sync_yandex_raw.py` pulls one or more **public** Yandex.Disk folder
links (no account/credentials needed — see `YANDEX_PUBLIC_LINKS` in `.env`)
one-way into `VAULT_PATH/raw/<folder-name>/...`. It:

- Downloads new/changed files only (compares Yandex's reported md5 against
  a local state file at `state/yandex_raw_state.json`, outside the vault).
- Archives (never hard-deletes) local copies of files removed upstream,
  moving them under `_trash/raw/...` like `archive_note` does.
- Maintains `raw/_status.md` — each file is `не прочитан` (new), `прочитан`
  (fetched via the `get_unread_files` tool), or `изменён с момента
  прочтения` (changed upstream after being marked read). Existing
  `прочитан` markers are preserved across runs unless the underlying file
  actually changed.

The agent reads through this queue with the `get_unread_files(limit)` MCP
tool (see "Tools exposed" above) — it pulls files by status out of
`_status.md` and marks them read in the same call. `app/extract.py` (pure,
no MCP dependency, used by `vault.get_unread_files`) does best-effort
extraction so the actual formats under `raw/` are usable, not just
flagged unreadable:

- **PDF** — real text extraction first; if that comes back essentially
  empty (a scan, or a CAD/vector export with no text layer — common for
  the architectural drawings in this vault), renders up to
  `PDF_MAX_PAGES_AS_IMAGES` pages as images instead.
- **DOCX** — paragraph and table text via `python-docx`.
- **Photos** (jpg/png/gif/webp/...) — resized and recompressed (JPEG,
  `IMAGE_MAX_DIMENSION`/`IMAGE_JPEG_QUALITY` in `app/extract.py`) and
  returned as an actual MCP image content block, not a text description —
  the model looks at it directly (native vision), no OCR involved.
- Anything else that isn't UTF-8 text (video, CAD/BIM formats like
  `.bimx`/`.dwg`, spreadsheets) still comes back with an `error` instead
  of content, but is still marked read.

`get_unread_files` caps total images per call at `MAX_IMAGES_PER_CALL`
(6 by default, in `app/vault.py`) — several embedded images add up fast
against the ~150k-character tool-result ceiling on claude.ai/Desktop. A
file that would push past that cap is left unread for a later call rather
than dropped, unless it's the very first result in the batch (so a call
always makes progress even when one file alone exceeds the soft cap).

`_status.md` has two independent writers — the sync script and
`get_unread_files` — so both go through `vault.locked_status_file` (a
`flock`-based lock) and re-read the file fresh before writing, rather than
trusting a possibly-stale in-memory copy. This matters because a backfill
run can take a long time (see below) while the agent may still be calling
`get_unread_files` on files the sync isn't touching.

Because it's a **public** link, anyone who has the URL can read that
folder's contents, no login required — treat the links themselves as
secrets (don't paste them anywhere public).

Downloads run concurrently (`MAX_CONCURRENT_DOWNLOADS` in the script, 30 by
default) — Yandex's public download endpoint has a fixed ~60-90s per-file
latency before it starts streaming bytes, regardless of size, so a
sequential backfill of ~1000+ files would take the better part of a day.
Progress checkpoints (state + status table) every `CHECKPOINT_EVERY`
completions (20 by default), so an interrupted run doesn't have to
re-download everything already fetched.

Runs on a schedule via `deploy/yandex-raw-sync.{service,timer}` (systemd
`--user` timer, every 15 minutes):

```bash
cp deploy/yandex-raw-sync.service deploy/yandex-raw-sync.timer ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now yandex-raw-sync.timer
journalctl --user -u yandex-raw-sync -f   # watch a run
```

Run it once by hand to backfill immediately instead of waiting for the
timer: `systemctl --user start yandex-raw-sync`.

## Notes on auth

The server is its own minimal OAuth 2.1 authorization server
(`app/oauth_provider.py`), not just a resource server checking someone
else's tokens. It registers exactly one pre-shared client — identified by
`OAUTH_CLIENT_ID`/`OAUTH_CLIENT_SECRET` from `.env` — and auto-approves
every `/authorize` request without a login screen. This is intentional for
a single-company internal tool: the actual security boundary is knowing the
client secret (kept by whoever adds the connector in claude.ai), exactly as
it was with the plain static bearer token this replaced. PKCE, redirect_uri
matching, client-secret verification, and access-token expiry are all
enforced by the `mcp` SDK itself — `oauth_provider.py` only stores and
retrieves codes/tokens (in memory; restarting the service invalidates
issued tokens, so anyone connected has to click "Connect" again).

If you ever need real per-user login (rather than one shared credential per
company), swap `StaticClientOAuthProvider` for a provider that redirects to
a real identity provider (Google Workspace, Microsoft Entra, etc.) in
`authorize()` — the rest of the server (`tools.py`, `vault.py`, the MCP
wiring) doesn't need to change.