# Obsidian MCP Wrapper

Obsidian MCP Wrapper is a session-stateless MCP retrieval system for an agency
Obsidian vault. It gives Claude Desktop users controlled access to indexed
Markdown knowledge without giving the model direct, unbounded filesystem access
or unsupervised write privileges.

The runtime has two layers:

- `pf-mcp`: a lightweight STDIO MCP shim that runs on each Claude Desktop
  workstation.
- `pf-index`: a shared LAN/VPN retrieval service that owns the vault index,
  signed handles, bearer-token authentication, rate limits, write locking, and
  the audit log.

The design goal is progressive disclosure: search returns compact snippets and
signed section handles first, then Claude expands only the context it needs.

## Highlights

- Standard-library core runtime for the default HTTP server path.
- Optional FastAPI/uvicorn ASGI boundary.
- SQLite FTS5 lexical search over immutable vault index epochs.
- HMAC-signed section handles that resolve against retained snapshots.
- Manifest-only filesystem access for indexed Markdown paths.
- Human-gated append writes into client, provider, carrier, and operational
  notes.
- Autonomous AI notes routed to a rotated capture inbox by default.
- File locking and SQLite audit records for write operations.
- Sanitized error envelopes that avoid leaking paths, tokens, URLs, or
  tracebacks into Claude.
- Quick-start, deployment, teardown, and nightly-refresh scripts.

## Runtime Topology

```text
Claude Desktop
  -> STDIO MCP JSON-RPC
  -> pf-mcp workstation shim
  -> HTTP POST /tools/<tool_name>
  -> pf-index LAN/VPN service
  -> Obsidian_Vault Markdown snapshots and .pf_index artifacts
```

`pf-index` consumes Markdown and derived index artifacts. It does not need the
source DuckDB database at runtime.

## Repository Layout

```text
.
|-- Docs/
|   |-- ARCHITECTURE.md
|   |-- DEPLOYMENT.md
|   |-- QUICKSTART.md
|   `-- MCP_WRAPPER_MASTER_PROMPT.md
|-- config/
|   |-- claude_desktop_config.example.json
|   |-- deployment_manifest.example.json
|   |-- pf-index.example.toml
|   `-- pf-mcp.env.example
|-- pf_index/
|   |-- api/                 HTTP boundaries
|   |-- core/                indexing, retrieval, crypto, paths, writes
|   `-- config.py            TOML and environment configuration
|-- pf_mcp/
|   `-- main.py              STDIO MCP shim
|-- scripts/
|   |-- build_index.py
|   |-- run_pf_index.py
|   |-- quickstart.sh
|   |-- quickstart_teardown.sh
|   |-- nightly_refresh.sh
|   |-- deploy_interactive.sh
|   `-- deploy_teardown.sh
|-- tests/
|-- pyproject.toml
`-- requirements.lock
```

## MCP Tools

The shim exposes these tools to Claude:

| Tool | Purpose |
| --- | --- |
| `vault_search` | Search indexed vault sections and return snippets with signed handles. |
| `strategy_search` | Search strategy, advisory, sales, renewal, and operating-system material. |
| `find_note` | Find likely note paths by query. |
| `get_note_summary` | Return a bounded summary/outline for one note path. |
| `get_entity_context` | Entity-oriented search wrapper. |
| `search_tables` | Search indexed table-like content. |
| `read_section` | Resolve one signed handle into a larger section excerpt. |
| `expand_context` | Resolve several signed handles within a token budget. |
| `get_full_profile` | Retrieve full materialized context for one known client or provider. |
| `advisory_context` | Combine profile evidence with strategy context for planning questions. |
| `vault_stats` | Return index and vault statistics. |
| `append_human_note` | Append a factual note through the gated write pipeline. |
| `remove_note` | Remove a specific generated note entry by note id. |

## Security Model

Important boundaries:

- The MCP shim validates argument shapes and hard limits before forwarding to
  the LAN service.
- The LAN service requires `Authorization: Bearer <token>`.
- Tokens map to user ids for audit and idempotency.
- Runtime reads are limited to paths listed in the active index manifest.
- Path validation rejects absolute paths, traversal, null bytes, URL schemes,
  Windows drive paths, and backslash paths.
- Section handles contain path, line range, content hash, file hash, vault id,
  and index epoch, then are signed with HMAC-SHA256.
- Old handles resolve against retained immutable snapshots, not changed live
  vault files.
- Autonomous writes are redirected to the configured capture inbox unless the
  human explicitly directs a write to a specific note.
- Write operations are append-only/gated, locked, audited, and bounded by
  configured note length limits.

## Requirements

- Python 3.11 or newer
- Markdown Obsidian vault directory
- HMAC secret for handle signing
- One or more bearer tokens for users

The default `pf_index.api.server` HTTP boundary uses only the Python standard
library. Optional FastAPI dependencies are listed in `requirements.lock`:

```bash
python -m pip install -r requirements.lock
```

## Quick Start

For a complete local smoke test, use the maintained guide:

```bash
sed -n '1,240p' Docs/QUICKSTART.md
```

The short version:

```bash
export PF_APP_DIR="$PWD"
export PF_DEMO_ROOT=/tmp/pf-mcp-demo
export PF_VAULT_ROOT="$PF_DEMO_ROOT/Obsidian_Vault"
export PF_INDEX_ROOT="$PF_DEMO_ROOT/.pf_index"
export PF_AUDIT_DB="$PF_INDEX_ROOT/write_audit.sqlite"
export PF_INDEX_HMAC_SECRET="local-demo-secret-change-me"
export PF_INDEX_TOKEN="local-demo-token"
export PF_INDEX_CONFIG="$PF_DEMO_ROOT/pf-index.toml"
```

Create a small test vault and config as shown in `Docs/QUICKSTART.md`, then
build an index:

```bash
PYTHONPATH="$PF_APP_DIR" python -m scripts.build_index \
  --config "$PF_INDEX_CONFIG" \
  --epoch 2026-06-08T120000Z
```

Run the stdlib HTTP server:

```bash
PYTHONPATH="$PF_APP_DIR" \
PF_INDEX_HMAC_SECRET="$PF_INDEX_HMAC_SECRET" \
python -m pf_index.api.server \
  --host 127.0.0.1 \
  --port 8765 \
  --config "$PF_INDEX_CONFIG"
```

Search through the API:

```bash
curl -s \
  -H "Authorization: Bearer $PF_INDEX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query":"Coverage","max_results":3,"max_tokens":1000}' \
  http://127.0.0.1:8765/tools/vault_search \
  | python -m json.tool
```

List MCP tools through the shim:

```bash
printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}' \
  | PYTHONPATH="$PF_APP_DIR" \
    PF_INDEX_URL="http://127.0.0.1:8765" \
    PF_INDEX_TOKEN="$PF_INDEX_TOKEN" \
    python -m pf_mcp.main \
  | python -m json.tool
```

Or run the automated path:

```bash
bash scripts/quickstart.sh
```

Tear down demo artifacts:

```bash
bash scripts/quickstart_teardown.sh
```

## Configuration

Server-side config lives in TOML. Start from:

```bash
cp config/pf-index.example.toml pf-index.toml
```

Key sections:

- `[paths]`: `vault_root`, `index_root`, and `audit_db`
- `[limits]`: result, token, note length, lock timeout, and concurrency limits
- `[security]`: `vault_id` and HMAC secret source
- `[tokens]`: bearer token to user id map
- `[inbox]`: rotated inbox path for autonomous captures
- `[retention]`: retained immutable index epochs

Workstation shim config is environment-based. Start from:

```bash
cp config/pf-mcp.env.example pf-mcp.env
```

Claude Desktop config example:

```bash
cat config/claude_desktop_config.example.json
```

## Indexing

Build an immutable index epoch:

```bash
PYTHONPATH="$PWD" python -m scripts.build_index --config /path/to/pf-index.toml
```

Each epoch contains:

```text
.pf_index/builds/<epoch>/
|-- manifest.json
|-- lexical.sqlite
`-- content/<sha256-of-relative-path>.md
```

The active epoch is selected through `.pf_index/ACTIVE`. Rolling back is a
controlled edit of that pointer to a retained epoch.

## Running The Service

Default stdlib server:

```bash
PYTHONPATH="$PWD" python -m pf_index.api.server \
  --host 127.0.0.1 \
  --port 8765 \
  --config /path/to/pf-index.toml
```

Optional FastAPI boundary:

```bash
python -m pip install -r requirements.lock
PYTHONPATH="$PWD" uvicorn pf_index.api.fastapi_app:create_app --factory \
  --host 127.0.0.1 \
  --port 8765
```

## Testing

Run acceptance and structure tests:

```bash
PYTHONDONTWRITEBYTECODE=1 python -m unittest discover -s tests -v
```

The tests focus on safety contracts such as path validation, signed handles,
indexing behavior, write routing, and repository structure.

## Deployment

Use `Docs/DEPLOYMENT.md` for the production runbook. It covers:

- Dedicated service user and filesystem permissions
- Code and vault placement
- HMAC secret generation
- User token generation
- Index build and startup checks
- `systemd` service setup
- LAN/VPN binding and firewall posture
- Runtime egress denial
- Token rotation
- Audit recovery
- Rollback

`scripts/deploy_interactive.sh` and `scripts/deploy_teardown.sh` support guided
deployment and cleanup flows.

## Nightly Refresh

`scripts/nightly_refresh.sh` supports a scheduled flow:

```text
sync or refresh source data
  -> optionally rebuild Markdown vault from DuckDB
  -> build a new immutable index epoch
  -> atomically update ACTIVE
```

If DuckDB-to-Markdown generation happens elsewhere, omit `PF_SOURCE_DATABASE`
and rebuild the index from the current vault.

## Data Boundary

Do not commit or deploy local data artifacts:

- `agency_core.duckdb`
- Generated `Obsidian_Vault/`
- `.pf_index/` builds
- `.obsidian/`
- `.vault_salt`
- Real `.env` files
- Audit databases
- Exported CSVs, spreadsheets, or reports

The repository is meant to contain application code, tests, docs, and placeholder
configuration only.

## Further Reading

- `Docs/ARCHITECTURE.md` for the full design and safety model
- `Docs/QUICKSTART.md` for local smoke testing
- `Docs/DEPLOYMENT.md` for production operations
- `Docs/MCP_WRAPPER_MASTER_PROMPT.md` for operator-facing MCP usage guidance