obsidian-second-brain

An Obsidian vault that an AI assistant can file into and search — self-hosted, plain Markdown on disk, no third party holding the notes.

Say "file this under Work" to Claude on your phone and a properly formed note lands in a vault on your own server, front matter filled in. Ask "what did I work out about chunking strategies" six months later and get it back, even though no note contains that phrasing.

The clever part isn't the AI. It's that filing costs almost nothing, so you actually do it. The vector index is what makes a pile of notes you never re-read still worth having.

Contents

• How it works

• The MCP tools

• The /pub read façade

• POST /ingest

• Requirements

• Quick start

• Configuration

• Vault conventions

• Design notes

• Security

• Development

Related MCP server: Obsidian MCP Server

How it works

Three containers share one vault directory. Two dependencies live outside: a Postgres with pgvector, and any OpenAI-compatible embeddings endpoint.

  Claude apps                        ┌─────────────────────────────────────┐
  (desktop, mobile,                  │  host                               │
   Claude Code)                      │                                     │
        │                            │   ┌───────────────┐                 │
        │  MCP over HTTPS            │   │  cloudflared  │  optional       │
        │  + bearer token            │   └───────┬───────┘                 │
        ▼                            │           │ :7620                   │
  ┌──────────────┐   outbound tunnel │           ▼                         │
  │  Cloudflare  │◄──────────────────┼── ┌──────────────────┐              │
  │  (optional)  │   no inbound port │   │ second-brain-mcp │──┐ rw        │
  └──────────────┘                   │   └──────────────────┘  │           │
                                     │                         ▼           │
                                     │   ┌──────────────────┐ ┌──────────┐ │
                                     │   │     indexer      │─┤  vault/  │ │
                                     │   │  polls every     │ │  *.md    │ │
                                     │   │  300s            │ └──────────┘ │
                                     │   └────────┬─────────┘      ▲       │
                                     │            │                │ rw    │
                                     │            │        ┌───────┴─────┐ │
                                     │            │        │  Obsidian   │ │
                                     │            │        │  (your app, │ │
                                     │            │        │   synced)   │ │
                                     │            │        └─────────────┘ │
                                     └────────────┼──────────────────────  ┘
                                                  │
                            ┌─────────────────────┴──────────────┐
                            ▼                                    ▼
                  ┌───────────────────┐              ┌──────────────────────┐
                  │ Postgres+pgvector │              │  embeddings endpoint │
                  │  chunks + vectors │              │  OpenAI-compatible   │
                  └───────────────────┘              └──────────────────────┘

Filing is decoupled from indexing. file_note returns as soon as the file is on disk. The note is findable by structured search immediately and by semantic search within one indexer cycle. A wedged embedding backend never slows down anything you are waiting on.

The indexer scans the vault rather than waiting to be told, so notes you type by hand in Obsidian are indexed exactly like notes the assistant files. That matters more than it sounds — most vaults end up mostly hand-written.

The MCP tools

Five tools over MCP's streamable-HTTP transport.

Three of the five never leave the filesystem. That is deliberate: filing and keyword search keep working when the model server is down, which on a homelab it regularly is.

Plus three plain HTTP routes for monitoring:

The /pub read façade

Some Claude deployments — a managed work tenant, for instance — do not allow custom connectors at all. The /pub routes are a read-only fallback that rides on an ordinary web fetch instead.

All GET, all returning text/markdown, all under /pub/<token>/:

Markdown rather than JSON, because it survives a fetch tool intact and reads correctly to a model on the far side. Responses carry Cache-Control: no-store (a capability URL in a shared cache is a leaked vault) and X-Robots-Tag: noindex, nofollow.

Set PUBLIC_READ_TOKEN to enable. Leave it unset and these routes return 404 — a prober learns nothing about whether they exist.

The handlers call the same functions the MCP tools call. They are a second transport, never a second implementation: a divergent search would show up as two clients disagreeing about what is in the vault.

The token is in the URL. That is what makes it work with a fetch tool, and it is a real cost: the token appears in proxy logs, in any reverse-proxy access log, and in browser history. Treat it as a read-only, rotatable credential and nothing more. The service redacts /pub/<token> from its own logs; it cannot redact anyone else's.

POST /ingest

A write path for the same constrained clients, taking a write-scoped token in an Authorization header rather than in a URL.

POST /ingest
Authorization: Bearer sbt_...
Content-Type: application/json

{
  "title":   "What I learned about HNSW",
  "body":    "# ...markdown...",
  "segment": "Industry",
  "tags":    ["pgvector", "retrieval"],
  "idempotency_key": "some-stable-id"
}

The idempotency key means a poller that retries does not duplicate the note.

Tokens live in a SQLite store (TOKEN_DB_PATH), not in Postgres — authentication has to keep working when the database does not. They are prefixed sbt_, stored hashed, carry a scope (read < write < admin), and can be individually expired or revoked with a recorded last-used time.

Requirements

On syncing the vault to your devices: Obsidian Sync has no CLI and no headless mode, so if you use it, a real Obsidian has to run somewhere always-on. The linuxserver/obsidian image works for this. Self-hosted alternatives (LiveSync over CouchDB, Syncthing) avoid that entirely. Whatever you choose, run exactly one syncer — two over one vault is how you get conflict files and lost edits.

Quick start

git clone https://github.com/gmoorevt/obsidian-second-brain.git
cd obsidian-second-brain
cp .env.example .env

1. Apply the database schema.

psql "$DATABASE_URL" -f schema.sql

schema.sql declares vector(1024), which matches Qwen3-Embedding-0.6B. Change it to match your model or every insert fails. bge-m3 is 1024, OpenAI text-embedding-3-small is 1536, nomic-embed-text is 768.

2. Verify your embeddings endpoint — the dependency most likely to be subtly wrong:

curl -s http://localhost:8080/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"YOUR-MODEL","input":"hello"}' \
  | python3 -c 'import json,sys; print(len(json.load(sys.stdin)["data"][0]["embedding"]))'
# → must equal the N in vector(N)

3. Scaffold the vault.

./vault/scaffold-vault.sh ./data/vault

Idempotent — safe to re-run, creates only what is missing, never overwrites a note. Use --dry-run first if you like.

4. Fill in .env — at minimum MCP_BEARER_TOKEN (openssl rand -hex 32), DATABASE_URL, and EMBEDDING_ENDPOINT. Delete the cloudflared service from docker-compose.yml if you are not publishing the endpoint.

5. Start it.

docker compose up -d --build
curl -s http://127.0.0.1:7620/healthz     # → ok

6. Connect your client to https://your-host/mcp with the bearer token as Authorization: Bearer <MCP_BEARER_TOKEN>.

Round-trip test: "File a note in Projects titled Connector Test", then "search my Projects inbox".

Configuration

Vault conventions

Deliberately shallow — four segments, two subfolders each, nothing deeper. Depth is where vaults like this die.

Work/  Industry/  Projects/  Life/
  Inbox/       ← everything new lands here
  Notes/       ← reviewed and kept

Attachments/   images, PDFs, binaries
Daily/         quick capture; no front matter required
_templates/    capture templates

Every note in a segment carries exactly six fields. Daily/ is exempt — that folder is a zero-friction scratchpad and requiring structure there defeats it.

---
title: Human readable title
created: 2026-09-07
segment: Industry          # Work · Industry · Projects · Life
tags: [rag, obsidian]      # flat list, no parent/child hierarchies
source: claude-desktop     # which channel wrote it
status: inbox              # inbox · filed
---

Filenames are YYYY-MM-DD-slugged-title.md. Collisions take a numeric suffix; nothing is ever overwritten.

vault/conventions.yaml is the single source of truth for all of this, read by both the scaffolding script and the service so they cannot disagree about the segment list. It is duplicated at src/second_brain_mcp/conventions.yaml because the package ships with it — keep the two identical.

Review is optional, and saying so out loud is what makes this survive. status: inbox exists so a review pass is possible, not owed. A note that sits at inbox for a year is still fully searchable. A system that quietly guilts you is a system you abandon.

Design notes

The decisions that were not obvious, and why.

The indexer is a separate container from the server. Same image, different entrypoint. A slow or wedged embedding backend must not make filing slow, and the two failures need to be separately visible to monitoring.

The Postgres connection is lazy. Built on first use, not at import. If it were eager, a database outage would stop the whole service from starting — including file_note, which needs no database at all.

Auth runs before parameter validation and any filesystem access. FastMCP ships a StaticTokenVerifier whose own docstring says not to use it in production. The replacement compares with hmac.compare_digest so the check does not leak the token by timing, rejects anything under 32 characters at startup, and never logs the supplied value — logging a near-miss token puts a credential in the log file.

Chunking splits on Markdown headings first, then windows anything still too long at 2,000 characters with 200 of overlap. Each chunk keeps its heading, so a hit can say which section it came from.

embedding_model is recorded on every chunk. Swap models and your existing vectors become meaningless — cosine distance across two embedding spaces returns confident nonsense rather than an error. Recording the model turns a silent corruption into a visible mismatch in index_status.

Don't share the tunnel's network namespace. Giving cloudflared network_mode: service:second-brain-mcp looks tidier and lets you route to localhost. It also couples the lifecycles: restarting the service gives it a new namespace and leaves cloudflared attached to the dead one. Service reports healthy, cloudflared reports healthy, public endpoint returns 530. Reach it by container name over the compose network instead.

A stalled sync driver looks exactly like nothing being wrong. Filing succeeds, the assistant reports a path, and the note simply never reaches your phone. Monitor sync separately from the endpoint.

Security

Read this before exposing the service to the internet.

The endpoint writes files to disk from network input. Treat it accordingly.

1. Put an allowlist in front of it. If only one client should ever reach this hostname, restrict it at the edge to that client's egress range — a Cloudflare WAF custom rule, or equivalent. A bearer token alone is one layer, and one layer on a public write endpoint is thin. Anthropic publishes its egress range at https://platform.claude.com/docs/en/api/ip-addresses; verify it rather than copying a range out of a blog post, this one included.

2. Do not route /ingest publicly. If the client that posts to it runs on your own network, reach it over the LAN and block the path at the edge. A route that is not routed cannot be misconfigured into being public.

3. /pub tokens are capability URLs. Anyone holding one can read the whole vault. Rotate them, scope them read-only, and remember that your reverse proxy and CDN log full request paths even though this service does not.

4. Bind to loopback unless you need otherwise. BIND_ADDR defaults to 127.0.0.1. Setting a LAN address publishes every route on the LAN, /pub included.

5. The token store is not in the vault. Keep it that way — credentials must not sync to every device Obsidian is open on.

This project has had no external security review. It is a personal homelab tool that is useful enough to share, not a hardened product.

Development

pip install -e ".[dev]"
pytest

96 tests, no network or database required — the suite covers the token store, the public façade rendering, redaction, and clamping.

Layout:

License

MIT — see LICENSE.