# scratchbook-mcp

> A **codebase-memory** MCP server for AI coding agents (Claude Code and others):
> hybrid **semantic + BM25** code search, a structural outline, a work journal,
> and **gaze-and-think** deep reasoning — over a local [LanceDB](https://lancedb.com).
>
> 給 AI 編碼代理(Claude Code 等)的**程式碼記憶** MCP 伺服器:
> 混合**語意 + BM25** 的程式碼搜尋、結構大綱、工作日記,以及 **gaze-and-think**
> 深度推理 —— 全部建在本機 [LanceDB](https://lancedb.com) 上。

[![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](./LICENSE)
![Node](https://img.shields.io/badge/node-%E2%89%A520-brightgreen)

---

## Why / 為什麼

`grep`/`rg` only find what you can already **name**, and mostly within one
directory. When an agent works in a large or unfamiliar repo, it spends attention
locating code instead of understanding it. scratchbook gives the agent a
cross-directory, **meaning-based** index of the code, a lexical (exact-identifier)
arm fused in, a place to write notes it can recall later, and a way to think
deeply about a tangled subsystem in a clean, isolated session. It **augments**
grep/Read — it does not replace them.

`grep`/`rg` 只找得到你**叫得出名字**的東西,而且多半困在單一目錄。當代理在龐大或
陌生的 repo 裡工作,注意力都耗在「定位程式碼」而不是「理解它」。scratchbook 給代理
一份跨目錄、**以意義為基礎**的程式碼索引,融入一條詞彙(精確識別字)臂,一個可日後
回想的筆記空間,以及一種在乾淨隔離的 session 裡把糾結子系統想透的方法。它**增強**
grep/Read —— 不是取代它們。

## Features / 特色

- **Hybrid retrieval / 混合檢索** — semantic (embedding) search fused with lexical
  BM25 via Reciprocal Rank Fusion. Meaning-based recall **and** exact-identifier
  precision (a query for `resolveAnnotatedMap` surfaces the exact symbol the
  embedding alone would rank below near-synonyms). Each hit is tagged `sem` /
  `lex` / `lex+sem`.
  語意(向量)搜尋與詞彙 BM25 以 RRF 融合:兼得「概念召回」與「精確識別字」。每筆
  結果標記由哪一臂命中(`sem` / `lex` / `lex+sem`)。
- **Zero-dependency fallback / 零依賴降級** — no embedding service? Everything
  still works in **lexical-only mode** (BM25 + refs/events/outline/journal); the
  first index run after an embedder appears **auto-upgrades** to hybrid.
  沒有 embedding 服務也能用:自動進入 **lexical-only 模式**(BM25 + 交叉引用/outline/
  日誌照常),之後偵測到端點會**自動全量升級**成混合檢索。
- **Exact cross-reference / 精確交叉引用** — `scratchbook_refs` enumerates **every**
  definition and reference of an identifier: grep-shaped (LIKE prefilter + line-level
  regex), deliberately *not* BM25 — an FTS tokenizer splits `snake_case` into an
  OR-flood and top-k truncation silently drops sites. `scratchbook_events`
  cross-references string **event names** (`.emit`/`.on` + wrapper methods) into an
  *event → send sites / listen sites* table and flags one-sided events — the string
  edges a syntactic call graph can't see.
  `scratchbook_refs` 窮舉一個識別字的**所有**定義與引用:grep 形而非排序形(LIKE
  預過濾 + 行級 regex),刻意不走 BM25 —— FTS tokenizer 會把 `snake_case` 切成 OR
  洪水,top-k 截斷即靜默漏報。`scratchbook_events` 交叉比對字串**事件名**
  (`.emit`/`.on` 與包裝方法),產出「事件 → 發送端/監聽端站點」對照表並標記單邊
  事件 —— 這些字串邊是語法呼叫圖看不見的架構。
- **Structural outline / 結構大綱** — each file's imports + top-level symbols with
  line numbers, to grasp organisation fast.
  每個檔案的 imports + 頂層符號 + 行號,快速掌握專案組織。
- **Work journal + recall / 工作日記與回想** — write dated notes (decisions,
  findings, suspected bugs); recall them later by meaning, with an **age** shown
  so stale notes read as old.
  寫下有日期的筆記(決策、發現、可疑 bug);日後以語意回想,並顯示**年齡**讓過時
  筆記一眼看得出來。
- **gaze-and-think / 凝視與思考** — dump a tangled problem into a *board*, then
  launch a clean, isolated flow session that concentrates solely on it, reasons
  about the **relationships** between the pieces, writes inspectable intermediate
  artifacts, and converges on a synthesis you can persist into recall.
  把糾結的問題丟進一個 *board*,再啟動一個乾淨隔離的心流 session,專注思考各片段間
  的**關係**、產出可檢查的中間產物、收斂成綜合結論,並可持久化進 recall。
- **Control-char-safe snippets / 控制字元安全的片段** — invisible bytes (NUL, …)
  are rendered `cat -A`-style (`^@`) so they can't be misread as spaces.
  隱形位元組(NUL 等)以 `cat -A` 風格顯示(`^@`),避免被誤讀成空格。

## Architecture / 架構

```
AI agent ──stdio (JSON-RPC)──▶ src/server.mjs  (MCP, 11 tools)
                                     │
     ┌───────────────┬──────────────┼──────────────┬───────────────┬──────────────┐
     ▼               ▼              ▼              ▼               ▼              ▼
 indexer.mjs      store.mjs      embed.mjs      gaze.mjs        walk.mjs      refs.mjs
 walk→chunk       LanceDB        Ollama-style   clean `claude   git ls-files / exact xref:
 →embed→upsert    3 tables +     /api/embed     -p` flow        gitignore-aware LIKE + line
 (incremental)    FTS (BM25)     (bge-m3, 1024) session         chunk.mjs      regex; event
                  ~/.scratchbook                                (symbol-aware)  send/listen
```

- **Runtime / 執行環境**: Node ESM, no build step (`node src/server.mjs`).
- **Storage / 儲存**: one LanceDB at `~/.scratchbook/lancedb`, every row namespaced
  by the canonical (realpath) project root — **one server serves every project**.
  單一 LanceDB,每列以正規化的專案根路徑命名空間 —— **一個伺服器服務所有專案**。
- **Tables / 資料表**: `chunks` (symbol-aware code chunks + vectors + a BM25 FTS
  index), `files` (per-file imports + outline), `journal` (dated notes + vectors +
  FTS).
- **Enumeration / 列舉檔案**: `git ls-files` in a git repo (perfect `.gitignore`
  respect); a manual walk with built-in ignores otherwise. Binaries, lockfiles,
  minified, and >512 KB files are skipped.
- **Incremental / 增量**: unchanged files skipped by `(mtime, size)` then content
  hash; only changed files are re-read and re-embedded.

## Requirements / 需求

- **Node ≥ 20** — that's all. Everything else is optional.
  只需要 **Node ≥ 20**,其他都是可選的。

## Quick start / 一鍵安裝

One command — no clone, no embedding service, works immediately:
一行指令 —— 不用 clone、不用 embedding 服務,立刻可用:

```bash
claude mcp add scratchbook --scope user -- npx -y github:jinyaolin/scratchbook
```

Out of the box scratchbook runs in **lexical-only mode**: full-text (BM25)
search, exact cross-reference (`refs` / `events`), outline, and the journal all
work with zero external services. Semantic search switches on automatically once
an embedder is reachable (next section).
開箱即是 **lexical-only 模式**:BM25 全文檢索、精確交叉引用(`refs` / `events`)、
outline、日誌全部可用,零外部服務。偵測到 embedding 端點後,語意搜尋自動開啟(見下節)。

Remove with `claude mcp remove scratchbook -s user`.

### Optional: semantic search / 可選:語意搜尋

Point scratchbook at any Ollama-style embedding endpoint — default is a local
Ollama with `bge-m3` (1024-dim):
給 scratchbook 一個 Ollama 風格的 embedding 端點即可 —— 預設是本機 Ollama 的
`bge-m3`(1024 維):

```bash
# https://ollama.com
ollama pull bge-m3
# Ollama serves the embedding API at http://localhost:11434/api/embed
```

The next `scratchbook_index` detects the embedder and **auto-upgrades** the
project (full re-embed); hybrid semantic+BM25 search takes over from there.
Use `SCRATCHBOOK_EMBED_URL` / `_MODEL` / `_DIM` for a remote box or another model.
下一次 `scratchbook_index` 偵測到端點就會**自動升級**(全量重嵌入),之後就是語意+BM25
混合搜尋。遠端機器或其他模型用 `SCRATCHBOOK_EMBED_URL` / `_MODEL` / `_DIM` 指定。

### Install from a clone / 從 clone 安裝

```bash
git clone https://github.com/jinyaolin/scratchbook.git
cd scratchbook
npm install

claude mcp add scratchbook --scope user \
  -- node /absolute/path/to/scratchbook/src/server.mjs
```

> Any MCP client works — the server speaks stdio JSON-RPC. See your client's
> docs for how to register a local stdio MCP server.
> 任何 MCP 客戶端都行 —— 伺服器走 stdio JSON-RPC。註冊方式見你客戶端的文件。

## Tools / 工具

| Tool | Purpose / 用途 |
|---|---|
| `scratchbook_index` | Index/refresh a codebase (incremental). Run once per project, again after big edits. / 索引/刷新一個 codebase(增量)。 |
| `scratchbook_search` | Hybrid (semantic + BM25) code search → `file:line`, symbol, snippet, `via`. / 混合檢索。 |
| `scratchbook_refs` | Exact, exhaustive who-uses-X: every definition + reference line of one identifier (case-sensitive). / 精確窮舉某識別字的所有定義與引用行。 |
| `scratchbook_events` | String event-name cross-reference: event → send / listen sites, one-sided events flagged ⚠. / 字串事件名交叉引用表,單邊事件標 ⚠。 |
| `scratchbook_outline` | Structural map: each file's imports + top-level symbols with line numbers. / 結構大綱。 |
| `scratchbook_journal` | Write a dated work-note (decisions, findings, suspected bugs). / 寫工作日記。 |
| `scratchbook_recall` | Hybrid search over past notes, with note age shown. / 回想筆記(顯示年齡)。 |
| `scratchbook_status` | Embedding health + indexed projects with counts. / 健康檢查與已索引專案。 |
| `scratchbook_forget` | Drop a project's code index (journal kept). / 移除某專案的程式碼索引(日記保留)。 |
| `scratchbook_gaze` | Write to a board and/or launch a flow session over it. / 寫入 board 並/或啟動心流 session。 |
| `scratchbook_gaze_read` | Read a board (notebook, sessions, artifacts); `remember=true` persists a synthesis into recall. / 讀 board;`remember=true` 持久化結論。 |

Every tool takes an absolute `project`/`path`; it defaults to the server's cwd.
每個工具都可傳絕對 `project`/`path`,預設為伺服器的 cwd。

## gaze-and-think / 凝視與思考

For a problem too tangled to hold in the working session, dump the pieces into a
**board** (a notebook), then launch a **flow session** — a fresh, isolated
`claude -p` that carries no other context and concentrates *only* on that board.
It reasons about the **relationships** between the pieces, writes intermediate
artifacts under the board's `artifacts/`, and converges on a synthesis. Runs in
the background; poll with `scratchbook_gaze_read`.

當問題糾結到裝不進當下的 session,把片段丟進一個 **board**(筆記本),再啟動一個
**心流 session** —— 一個乾淨隔離、不帶其他上下文、只專注在該 board 的 `claude -p`。
它思考各片段的**關係**、把中間產物寫進 board 的 `artifacts/`、收斂成綜合結論。背景
執行,用 `scratchbook_gaze_read` 取回。

```
scratchbook_gaze  board="X" note="a hard piece…"     # accumulate material / 累積素材
scratchbook_gaze  board="X" focus="how do these relate?" project="/abs/repo"
scratchbook_gaze_read board="X"                       # read synthesis + artifacts
scratchbook_gaze_read board="X" remember=true         # persist synthesis → recall
```

The flow session runs with `--permission-mode acceptEdits` and read/write/search
tools **but no shell** — it can build checkable artifacts and ground claims in
real code, but never runs arbitrary commands. A remembered synthesis is stamped
with the commit it was grounded in and its age is shown on recall, because it has
no source file to re-derive from and so does **not** auto-update — treat it as
understanding *as of* that commit, and verify against current code.

心流 session 以 `--permission-mode acceptEdits` 執行,有讀/寫/搜尋工具但**無 shell**
—— 能產出可檢查的產物、對真實程式碼 grounding,但不執行任意指令。被 remember 的
綜合結論會蓋上「當時所依據的 commit」戳記,recall 時顯示年齡:它沒有源檔可重新推導、
**不會**自動更新,請當成「截至該 commit 的理解」,並對照現行程式碼驗證。

> Requires the `claude` CLI on `PATH` for the flow session.
> 心流 session 需要 `PATH` 上有 `claude` CLI。

## CLI (manual use / debugging) / 命令列(手動使用 / 除錯)

```bash
node src/cli.mjs index  /path/to/your/repo               # index a repo
node src/cli.mjs index  /path/to/your/repo --reindex     # force full re-embed
node src/cli.mjs search "where sessions are cached" --project /path/to/your/repo --full
node src/cli.mjs outline --project /path/to/your/repo --path src/lib
node src/cli.mjs status
node src/cli.mjs note "flagged N+1 in loader" --tags bug,perf --project /path/to/your/repo
node src/cli.mjs recall "what was I doing here" --project /path/to/your/repo
node src/cli.mjs forget --project /path/to/your/repo
```

## Configuration (env) / 設定(環境變數)

| Env | Default | Meaning / 意義 |
|---|---|---|
| `SCRATCHBOOK_DB` | `~/.scratchbook/lancedb` | LanceDB directory / LanceDB 目錄 |
| `SCRATCHBOOK_EMBED_URL` | `http://localhost:11434/api/embed` | Ollama-style embed endpoint / embed 端點 |
| `SCRATCHBOOK_EMBED_MODEL` | `bge-m3` | embedding model / 模型 |
| `SCRATCHBOOK_EMBED_DIM` | `1024` | vector dimension (must match model) / 向量維度(須與模型一致) |
| `SCRATCHBOOK_EMBED_BATCH` | `16` | texts per embed request / 每次請求文本數 |
| `SCRATCHBOOK_EMBED_CONCURRENCY` | `4` | parallel embed requests / 並行請求數 |
| `SCRATCHBOOK_GAZE_DIR` | `<db-dir>/../gaze` | gaze boards directory / gaze board 目錄 |
| `SCRATCHBOOK_MAX_FILE_KB` | `512` | skip files larger than this / 略過超過此大小的檔 |
| `SCRATCHBOOK_CHUNK_MAX_CHARS` | `1600` | max chunk size / chunk 上限 |
| `SCRATCHBOOK_CHUNK_MIN_CHARS` | `240` | min before a soft cut / chunk 下限 |
| `SCRATCHBOOK_CHUNK_OVERLAP_LINES` | `8` | overlap when window-splitting / 視窗切分重疊行數 |

## Notes / limits / 注意事項與限制

- The embedding **dimension is baked into** the LanceDB schema. Changing to a
  model of a different dimension means a fresh DB (delete `~/.scratchbook/lancedb`).
  embedding **維度寫死進** LanceDB schema;換到不同維度的模型需重建 DB(刪掉該目錄)。
- A recalled note / remembered gaze synthesis reflects the code **as it was when
  written** and does not auto-update — it's a strong prior, not a substitute for
  reading current code.
  回想到的筆記 / 記憶的 gaze 結論反映**寫入當下**的程式碼、不會自動更新 —— 是強力
  的前導,不是「讀現行程式碼」的替身。
- Chunking is heuristic (regex anchors), not a full parser — good enough for
  retrieval and outlines, not a semantic analyser.
  切塊是啟發式(regex 錨點),不是完整 parser —— 對檢索/大綱夠用,不是語意分析器。
- Data under `~/.scratchbook/` is fully rebuildable from source — safe to delete.
  `~/.scratchbook/` 下的資料完全可從原始碼重建 —— 可安心刪除。

## License / 授權

[Apache License 2.0](./LICENSE) — Copyright 2026 Jinyao Lin.