MAST
README.md
# MAST — Monorepo AST Search Tool
MAST is a code-search engine that runs as either an MCP server (for AI assistants) or a standalone CLI. It parses TypeScript and JavaScript source files with a real AST parser (`tree-sitter`), stores the resulting symbol graph and code chunks in SQLite, and answers queries with lexical BM25 search fused against a declaration-exact ranker via Reciprocal Rank Fusion.
The core design principle: **return exactly the code an assistant needs, nothing more**. Rather than reading entire files, MAST returns the specific function, interface, or type declaration that matches a query — saving tokens, reducing context noise, and letting AI tools navigate large codebases without drowning in irrelevant content.
---
## Contents
- [Why MAST?](#why-mast)
- [Requirements](#requirements)
- [Install](#install)
- [Quick Start](#quick-start)
- [Use it from your AI assistant](#use-it-from-your-ai-assistant)
- [Upgrading](#upgrading)
- [Using MAST in a monorepo](#using-mast-in-a-monorepo)
- [CLI Reference](#cli-reference)
- [MCP Tool Reference](#mcp-tool-reference)
- [Configuration](#configuration)
- [How It Works](#how-it-works)
- [Token Efficiency](#token-efficiency)
- [History](#history)
---
## Why MAST?
When an AI assistant needs to understand code, the naive approach is to read full files. This wastes tokens (most of a 200-line file is irrelevant to the question), inflates context windows, and forces the model to filter signal from noise on every call.
MAST takes a different approach:
- **AST-level chunking** — every function, class, interface, and type alias is its own chunk. The assistant gets the exact declaration it needs, not the file it happens to live in.
- **Ranked search** — BM25 (FTS5) handles keyword and identifier queries; a declaration-exact ranker ("ranker D") catches exact-symbol-name queries that BM25's trigram tokenizer can rank inconsistently. Both are fused via Reciprocal Rank Fusion so a chunk that both rankers agree on outranks one that only one of them found.
- **Structural queries** — "who calls this function?", "what implements this interface?", "what does this file import?" are answered from a pre-built symbol graph, not by grepping source. Answers are instantaneous and structurally correct.
- **Staleness handling that says what it actually does** — five read tools (`mast_signature`, `mast_callers`, `mast_exports`, `mast_dependencies`, `mast_rename_impact`) re-parse a changed file inline before answering; `mast_search` and `mast_implementors` flag affected results with `stale: true` rather than re-parsing. Both only cover files the index *already knows*: a brand-new file is invisible until a reindex, which is why `mast serve` watches by default and why `mast_search` carries an `unindexed_files` warning when it does not.
- **Token accounting** — every tool response includes `_stats` with the token count returned and the counterfactual "what would a naive full-file read have cost?", giving a concrete measure of efficiency over time.
---
## Requirements
- **Node.js ≥ 22** (this repo pins the version it develops against in `.nvmrc`)
- **A C++ toolchain**, for the two native modules (`better-sqlite3`, `tree-sitter`).
Prebuilt binaries cover most platforms; when none matches your Node ABI, `node-gyp`
builds from source and needs:
- **macOS** — `xcode-select --install`
- **Debian/Ubuntu** — `sudo apt install build-essential python3`
- **Windows** — install the "Desktop development with C++" workload from Visual Studio
Build Tools
No services, no API keys, no network at query time. Everything is local SQLite.
---
## Install
As a dev dependency of the project you want to index — recommended, because the version
is then pinned in your lockfile alongside everything else:
```bash
pnpm add -D @spikedpunch/mast # or: npm i -D / yarn add -D
```
Or globally, if you want one `mast` across many checkouts:
```bash
pnpm add -g @spikedpunch/mast
```
Verify:
```bash
mast --version
```
---
## Quick Start
Three commands from nothing to a searchable index:
```bash
cd /path/to/your/project
mast init # write .mast/, then run the first full index
mast status # confirm it is fresh
mast search "createUser" # search it
```
`mast search` prints the matching declaration, not the file it lives in:
```
$ mast search "compareVersions" -n 1
src/cli/upgrade-cmd.ts:39 compareVersions function (exported)
/** Semver compare, prerelease-aware. Returns <0, 0, or >0. */
export function compareVersions(a: string, b: string): number {
...
}
270 tokens returned vs 2140 to read the files whole — 87% saved
```
The last line is real accounting, not a slogan: every response carries `_stats` with what
it returned and an upper bound on reading the referenced files whole. On a small file the
saving can be *negative*, and MAST says so rather than rounding it into a win.
An answer that MAST cannot fully stand behind says so, on the same surface that shows the
result. A file edited since it was indexed is marked, because the body printed under it is
the *old* one:
```
! 1 of 2 results are from files that changed since indexing —
the code shown below may be out of date. Run `mast index` to refresh.
src/a.ts:1 alphaFunction function (exported) [STALE]
```
And an empty answer distinguishes the two reasons it can be empty:
```
$ mast search "kept_symbol"
no matches (mast indexes TypeScript, JavaScript, and Markdown only, and skips dot
directories unless include_dot_dirs names them — a symbol anywhere else is
invisible to it, not absent from the repo)
$ mast search "anything" # in a directory with no index
nothing is indexed at this path — this is not evidence the symbol is absent.
run `mast index` first, or check `mast status` for the path being used.
```
Narrow it with `--type`, `--language`, `--exported`, `--file`, `-n`:
```bash
mast search "greet" --type method --exported -n 5
mast search "config" --file "src/store/**"
```
Keep it current as you work — or let a git hook do it:
```bash
mast index --incremental # reindex only what changed
mast install-hooks # reindex automatically after commits and checkouts
```
Then tell your agent to use it. One command installs what your harness can take — hooks
that prime the agent at session start and remind it before a built-in search, or a rules
file where there are no hooks ([details](#tell-the-assistant-how-to-use-it)):
```bash
mast setup claude # or cursor, vscode, windsurf, zed, desktop
```
Everything shipped with your build is readable offline, so you never have to work out
which docs match your version:
```bash
mast docs # list the topics
mast docs spec # the full behavioural specification
mast docs signals # how to read the flags on an answer (stale, truncated, empty)
mast skill # the instructions to paste into an agent prompt
```
---
## Use it from your AI assistant
MAST speaks MCP over stdio. `mast serve` is the server command; the configuration below
differs only in where each tool keeps its config file.
If you installed MAST as a dev dependency rather than globally, replace `mast` with
`npx @spikedpunch/mast` (or `pnpm exec mast`) in any of these.
### Claude Code
```bash
claude mcp add mast -- mast serve
```
Add `--scope project` to write `.mcp.json` into the repository so your team picks it up
from the checkout.
To prime the agent at session start and remind it before a search: `mast setup claude`.
### Git worktrees
A committed `.mcp.json` is used in every worktree of the repository, so it should start a
`mast` that every worktree can find. **Install mast globally** and use `"command": "mast"`.
A path into `node_modules` (`node node_modules/@spikedpunch/mast/dist/cli/index.js serve`)
fails in a new worktree until its dependencies are installed, and that session gets no mast
tools. Keep `--state-dir` relative (or leave it out), so each worktree gets its own index.
In a worktree the server builds that index but does not watch for changes; call
`mast_reindex` after creating or renaming files there.
```json
{
"mcpServers": {
"mast": { "type": "stdio", "command": "mast", "args": ["serve"] }
}
}
```
A subagent that Claude Code runs in a worktree still talks to its parent session's server,
which indexes the parent's checkout.
### Claude Desktop
`~/Library/Application Support/Claude/claude_desktop_config.json` on macOS,
`%APPDATA%\Claude\claude_desktop_config.json` on Windows:
```json
{
"mcpServers": {
"mast": {
"command": "mast",
"args": ["serve"],
"env": { "MAST_STATE_DIR": "/absolute/path/to/your/project/.mast" }
}
}
}
```
Claude Desktop does not run in your project directory, so `MAST_STATE_DIR` must be
absolute. The CLI and editor integrations below infer it from the working directory.
Claude Desktop has no hook system and no rules file mast can write. The instructions string
`mast serve` sends in the MCP handshake is its only channel, and it needs no setup;
`mast setup desktop` just says so.
### Cursor
`.cursor/mcp.json` in the project, or `~/.cursor/mcp.json` globally:
```json
{
"mcpServers": {
"mast": { "command": "mast", "args": ["serve"] }
}
}
```
To add the hooks and a rules file: `mast setup cursor`.
### VS Code (GitHub Copilot)
`.vscode/mcp.json`:
```json
{
"servers": {
"mast": { "type": "stdio", "command": "mast", "args": ["serve"] }
}
}
```
To add the session-primer hook: `mast setup vscode`.
### Windsurf
`~/.codeium/windsurf/mcp_config.json`:
```json
{
"mcpServers": {
"mast": { "command": "mast", "args": ["serve"] }
}
}
```
To add a rules file: `mast setup windsurf`.
### Zed
`settings.json`:
```json
{
"context_servers": {
"mast": { "command": { "path": "mast", "args": ["serve"] } }
}
}
```
To add mast's instructions to a rules file you already keep (`.rules`, `AGENTS.md`, and the
others Zed reads): `mast setup zed`.
### Any other MCP client
Run `mast serve` over stdio from the project root. It advertises eleven tools — ten that
read and `mast_reindex`, which writes — and needs no arguments beyond `serve`.
### Tell the assistant how to use it
Registering the server gives the model the tools; it does not tell it *when* to reach for
them. `mast setup <harness>` installs what each harness can take:
| harness | session primer | search reminder | static instructions |
|---|---|---|---|
| `claude` (Claude Code) | yes | yes, before `Grep` and `Glob` | none written |
| `cursor` | yes | yes, but after the search runs | `.cursor/rules/mast.mdc` |
| `vscode` (VS Code Copilot) | yes | no | none written |
| `windsurf` | no | no | `.windsurf/rules/mast.md`, or `.devin/rules/mast.md` if `.devin/` exists |
| `zed` | no | no | a marked block in the first rules file that exists |
| `desktop` (Claude Desktop) | no | no | none written |
The session primer is the rule set plus the live index state, given at session start. The
search reminder is one line naming `mast_search`, attached to a built-in search. Windsurf,
Zed and Claude Desktop have no hook system mast can use, so for them `mast setup` installs
the static file only, and says so. `mast serve` also sends a short instructions string when
the MCP connection opens; whether a harness passes that to the model is the harness's choice.
```bash
mast setup claude # project scope; add --global for ~/.claude/settings.json
mast setup cursor --check # exit 0 only if the hooks and the rules file are current
mast setup windsurf --dry-run
mast setup zed --remove
```
Anything else, or a harness not listed, can use `mast skill`, which prints the same short
instructions to paste into a system prompt, `CLAUDE.md`, or a skill file:
```bash
mast skill # print it
mast skill --install # splice it into this project's agent config files
mast skill --install --dry-run
```
`--install` writes only into files that **already exist** — `CLAUDE.md`, `AGENTS.md`,
`.cursorrules`, `.windsurfrules`, `.github/copilot-instructions.md` — and writes inside a
marked block, so re-running after an upgrade replaces the previous copy instead of adding
a second one. It never runs on its own, and it never creates a config file you did not
already keep.
**What is verified.** In Claude Code, both hooks were observed delivering their text to the
model in one session. For Cursor, VS Code and Windsurf, the hook formats and rules-file
locations were read from the vendor's documentation and nothing was run inside the tool.
The list of files Zed reads was not checked against Zed's documentation. Whether any of
this changes how often an agent uses mast has not been measured.
---
## Upgrading
```bash
mast upgrade
```
This checks for a newer release and prints the exact command for how *you* installed it —
it does not upgrade in place, because a CLI cannot reliably tell a global install from a
dev dependency, and guessing wrong runs the wrong command in your repository.
More importantly, it tells you the one thing your package manager cannot: whether the
upgrade changes the **index schema**. When it does, MAST empties the index and rebuilds
it on the next `serve` or `index`, and says so in one line. Until then `mast search` and
`mast query` stop with an error naming both versions, so an answer is never drawn from rows
another version wrote. Usage metrics are kept. An index built by a *newer* MAST is never
emptied: the older one stops and says to upgrade. Nothing is lost that cannot be rebuilt,
but on a large monorepo the rebuild is minutes, and it is better known in advance than
discovered as an unexplained stall.
If mast's hooks, rules files or skill blocks are installed in the project (or hook files
under your home directory), the report ends with an "After upgrading" block listing the
commands to re-run, such as `mast setup claude --check` and `mast skill --install`. It runs
on the version you have now, so it cannot know what the new version would write: it only
knows the files exist, and `--check` is what compares them. A project with none of these
gets no such block.
---
## Using MAST in a monorepo
**One index at the repository root** is usually right. Cross-package imports resolve, so
`mast_callers` finds callers in sibling packages — which is the reason to use a monorepo
tool rather than one index per package.
**What is indexed.** `.ts`, `.tsx`, `.js`, `.jsx`, and `.md`, minus `node_modules`,
`dist`, `build`, `coverage`, `.next`, `.turbo`, `.mast`, and test files. Override with
`--extensions` and `--exclude` on `mast init`, or — for a setting the whole team should
get — `file_extensions` / `exclude_patterns` in `mast.config.json` at the project root.
Editing `.mast/config.json` also works and is read back, but that file is gitignored and
per-machine, so the change will not travel; `mast.config.json` outranks it.
**Dot directories are skipped unless you name them.** A directory whose name starts with
a dot (`.agents`, `.github`, `.storybook`) is not walked, and neither is a dot-leading
file, even inside a directory you list. To index a dot directory, list it in `mast.config.json`:
```json
{ "include_dot_dirs": [".agents", "packages/app/.storybook"] }
```
Each entry is a directory path relative to the project root, not a glob. The directory is
walked recursively; a dot directory nested inside it needs its own entry, and
`exclude_patterns` still applies. It is opt-in because a dot directory is as likely to
hold editor history or a second checkout of your source (`.history`, `.claude/worktrees`)
as anything worth searching, and duplicates of real symbols make caller lists wrong. Run
`mast walk` to see what the config walks before you index, then `mast index`.
**Other languages are not indexed, and this matters.** MAST parses TypeScript and
JavaScript only. A symbol defined in Python, Go, Java, or Rust is absent from the index,
which looks exactly like absent from the repository. Treat an empty result as "MAST did
not find it", never as "it does not exist" — `mast skill` says this to the model too.
**Add `.mast/` to `.gitignore`.** It is derived state, it is large, and it is
machine-specific — on a 14k-file monorepo it is around 420 MB, almost all of it `graph.db`.
Ignore the whole directory, including `.mast/config.json`: that file is a *resolved*
snapshot and carries absolute paths (`project_root`, `resolved_state_dir`) that mean
nothing on anyone else's machine. The file meant to be committed is `mast.config.json` at
the project root — see the next paragraph.
**If you move the index, move the ignore rule with it.** `.mast/` is the default location,
not the only one: a `state_dir` in `mast.config.json`, a `MAST_STATE_DIR` in the
environment, or a `--state-dir` flag all put the index somewhere else, and a `.gitignore`
naming `.mast/` then protects nothing.
**A custom index location is not remembered between runs.** `--state-dir` applies to the
one command you pass it to. Path settings are deliberately never read back out of a
persisted config — an absolute path written by a previous run (or a previous container)
can resolve somewhere that no longer exists, or worse, somewhere belonging to a different
project. To make a custom location stick, put it in source control or the environment:
```json
// mast.config.json, at the project root
{ "state_dir": ".cache/mast" }
```
```bash
export MAST_STATE_DIR=/absolute/path/to/index
```
Resolution order is `--state-dir` → `MAST_STATE_DIR` → `mast.config.json` → `.mast`.
`mast status` prints the directory it resolved, and says so plainly when nothing has been
indexed there.
**Scale.** A cold index of VS Code — 8,653 files, 152,969 chunks — takes about two
minutes and produces a 794 MB state directory. Incremental reindexing of a changed file
is milliseconds.
## CLI Reference
### `mast init [path]`
Initialise MAST for a project and run the initial full index.
```
Options:
--state-dir <dir> Where to write index state (default: <path>/.mast)
--extensions <ext,...> File extensions to index (default: .ts,.tsx,.js,.jsx,.md)
--exclude <pattern,...> Glob patterns to exclude
--no-index Create config only; skip initial indexing
```
**Why:** Creates the state directory structure, writes `config.json`, and runs a full parse + symbol extraction pass. Running this once upfront means subsequent incremental runs only touch changed files.
---
### `mast search <query> [path]`
Search the index and print readable results.
```
Options:
-n, --limit <n> Max results, 1-50 (default: 10)
-t, --type <kind> function | method | class_shell | interface | type | export | block | doc
-l, --language <lang> typescript | javascript | markdown
-e, --exported Only exported symbols
-f, --file <glob> Restrict to files matching a glob
--state-dir <dir> State directory
--reindex Incrementally reindex before searching
--json Emit the raw MCP response instead of text
```
`--reindex` exists because the CLI has neither freshness mechanism the MCP server has:
no file watcher, and no cached freshness probe (a one-shot process has no lifetime to
amortise one over). Without it, a file created since the last index run is invisible —
JIT staleness only re-parses files the index already knows. It is opt-in because it adds
a whole incremental index run to what is otherwise a single query, and it reports what it
did on **stderr**, so `--json` consumers keep a parseable stdout. A reindex that loses
`structure.lock` to a concurrent writer — likelier now that `mast serve` watches by
default — warns and queries the existing index rather than failing the search.
**Why:** the fastest way to check what the index actually contains, and the same code path
the MCP `mast_search` tool uses — it dispatches through the registered handler rather than
re-implementing ranking, so CLI and assistant results cannot disagree. Staleness and
truncation flags are printed above the results; an empty result that is empty *because* the
index was busy says so.
For scripting, `mast query mast_search '{...}'` gives byte-identical MCP output.
---
### `mast index [path]`
Build or update the index.
```
Options:
--state-dir <dir> State directory
--incremental Only reindex files changed since last run
--show-progress Print indexing progress to stderr
--checker Opt-in TypeScript-checker pass: upgrades heuristic potential_matches
into verified caller edges (or drops non-call-site noise). Can take
minutes on a large monorepo — not part of the default path. Its
results last until the next index run that writes a file.
```
**Why incremental:** The incremental path diffs the current file manifest against stored mtimes. Only stale, added, or deleted files are processed — for a large codebase this cuts index time from seconds to milliseconds on most runs.
---
### `mast serve`
Start the MCP server over stdio.
```
Options:
--state-dir <dir> State directory
--no-startup-reindex Skip the startup staleness check (not recommended)
--no-watch Do not watch source files (batch/container use)
--watch Watch source files and incrementally reindex on change
(the default; accepted for compatibility)
```
Watching is **on by default**. JIT staleness only re-parses files the index already knows,
so without a watcher a file created during a session stays invisible to every read tool
until something reindexes — and nothing does. A watcher failure (EMFILE, permissions) logs
a warning and the server keeps serving, so the default cannot stop `serve` from starting.
In a git worktree (`git worktree add`) the watcher is off, even with `--watch`: each
worktree gets its own index, but a repository runs one watcher, not one per worktree.
Call `mast_reindex` after creating or renaming files in a worktree.
The server implements a four-step startup ladder so MCP clients get a usable server in under a second even for large projects. See [Startup Ladder](#startup-ladder) for details.
---
### `mast status [path]`
Print index health.
```
Options:
--state-dir <dir> State directory
--json Output as JSON
```
Reports `last_indexed`, `indexed_files`, `chunk_count`, `stale_files`, `parse_errors`, `write_errors`, `index_fresh`, and `freshness_cause`. When `stale_files` is non-zero it also names the stale files, up to 20 per category (changed, unindexed, deleted). Use this to diagnose why search results look outdated. It also prints `schema_version` (this binary's) and `index_schema_version` (the version that built the index); when they differ, `mast search` and `mast query` stop with an error until `mast index` has rebuilt it.
---
### `mast walk [path]`
Show which directories and files the config makes mast walk. It reads no index and writes
nothing.
```
Options:
--depth <n> Roll directories up to this many path segments
--files List every walked file instead of the directories
--state-dir <dir> State directory
--json Output as JSON
```
```
$ mast walk --depth 1
project_root: /path/to/project
config: /path/to/project/mast.config.json
saved config: /path/to/project/.mast/config.json (supplies any key mast.config.json does not set)
file_extensions: .ts .tsx .js .jsx .md
exclude_patterns: **/node_modules/** **/dist/** ...
include_dot_dirs: .agents .agnets
! .agnets: no directory of exactly this name under the project root. Check the spelling in include_dot_dirs, including upper and lower case.
158 files in 6 directories
10 .
2 .agents
...
```
Each line is a directory and the number of walked files in it (with `--depth`, in it and
below it). An `include_dot_dirs` entry that contributed no files is flagged, and the flag
says why: no directory of exactly that name (case counts, even on a filesystem where it
otherwise would not), a symbolic link (mast does not follow them), a file, or a real
directory with nothing walked below it.
The `saved config` line appears when `mast init` or `mast serve` has saved a config in the
state directory. That file supplies every key `mast.config.json` does not set, so an entry
you delete from `mast.config.json` keeps applying until you set the key explicitly (`[]`
for none). This is true of `exclude_patterns` and `file_extensions` as well.
**Why:** it is how you test a config change before paying for an index run. It calls the
same walk `mast index` and `mast status` use, so its answer is what would be indexed. A
config that mast rejects (a glob or an absolute path in `include_dot_dirs`, say), or a
path that is not a directory, is printed as one line on stderr with exit 1. Every other
command prints a rejected config the same way, as `mast: <message>`, and `mast index` and
`mast init` refuse a path that is not a directory instead of creating it.
---
### `mast metrics [path]`
Show token-efficiency metrics.
```
Options:
--since <window> Time window: 7d, 24h, 30m (default: 7d)
--rollup Collapse raw rows older than --keep-days into daily roll-ups
--vacuum Delete daily roll-up rows older than --keep-days
--keep-days <n> Retention days (default: 7 for rollup, 90 for vacuum)
--state-dir <dir> State directory
```
Prints a column-aligned table: tool name, call count, tokens returned, average duration, and efficiency ratio. Use `--rollup` + `--vacuum` periodically to keep the metrics database from growing unbounded.
---
### `mast install-hooks [path]`
Install git `post-commit` / `post-checkout` hooks that run `mast index --incremental` automatically, so the index stays fresh across commits and branch switches without a manual step.
---
### `mast query <tool> [json] [path]`
Invoke any MCP read tool directly, with byte-identical output to the MCP transport.
```
Options:
--state-dir <dir> State directory
--reindex Incrementally reindex before querying (see `mast search --reindex`)
--json Emit the exact single-line MCP response (default pretty-prints)
```
```bash
mast query mast_callers '{"symbol":"resolveConfig"}'
mast query mast_project_skeleton '{}'
```
**Why:** the scripting and debugging surface. `mast search` is the readable front door to
one tool; this reaches all eleven, and returns exactly what an assistant would receive —
so a disagreement between what you see and what the model saw is not possible. Naming a
tool that does not exist lists the ones that do.
---
### `mast docs [topic]`
Print documentation shipped with the installed build — `readme`, `spec`, `skill`, or
`signals` (the staleness, truncation and emptiness flags on an answer). No argument lists
the topics with the version they belong to.
**Why:** removes the step where a reader looks up their version and then finds docs for a
different one. What `mast docs` prints is what the binary in your `node_modules` does.
---
### `mast skill [path]`
Print the MAST instructions to paste into an agent prompt, `CLAUDE.md`, `.cursorrules`, or
a skill file.
```
Options:
--install Splice into this project's existing agent config files
--dry-run With --install, report what would change without writing
```
**Why:** registering the MCP server gives a model the tools but not the judgement — when to
search instead of reading, and that code tokens beat prose in a query. The text is short
(under 4,000 characters) so it fits a rules file. It also tells the model that an empty
result means "MAST did not find it", not "it does not exist", which is the single most
consequential thing to get right about a search tool, and points at `mast docs signals` for
how to read the individual flags.
---
### `mast prime [path]`
Print the session primer: a short rule set for using mast, plus the live index health.
```
Options:
--state-dir <dir> State directory
```
**Why:** it is the text an agent should be given at the start of a session. When the index
is fresh or stale it prints the rules and the file count and age of the index (a stale
index adds the changed/unindexed/deleted split and a `mast_reindex` instruction). When
there is no index, or the index describes a different tree, it withholds the rules and says
what is wrong. It exits 0 in every state, unlike `mast status`.
---
### `mast hook <harness> <event>`
The entry point for agent hooks. Reads the harness's hook JSON on stdin and writes that
harness's JSON envelope on stdout, or writes nothing. `<harness>` is `claude`, `cursor` or
`vscode`; `<event>` is `session-start` or `search`.
- `session-start` emits the same text `mast prime` prints for the project (the hook's `cwd`,
else the first workspace root, else the current directory), in every index state.
- `search` emits a one-line reminder to try `mast_search` first. It stays silent when no
index exists for the project, or when the search is scoped by `type`, `glob` or `path` to
a language mast does not index. "Indexes" follows the project's own `file_extensions`
where it sets one, so a project that indexes `.mjs` is reminded on a `*.mjs` search.
```
Arguments:
harness claude | cursor | vscode
event session-start | search
```
**Why:** a hook that fails breaks the user's session, so this command exits 0 for every
ordinary condition (empty or malformed stdin, an unknown harness or event, a missing index)
and leaves stdout empty, with one line on stderr where something went wrong. `search` runs
before every Grep, so it is dispatched before the rest of the CLI loads.
---
### `mast setup <harness> [path]`
Install what tells an agent to use mast: hooks that prime it at session start and remind it
before a built-in search, and rules files for harnesses that read them. `<harness>` is
`claude`, `cursor`, `vscode`, `windsurf`, `zed` or `desktop`; `[path]` is the project root
(default: the current directory).
| harness | project file (default) | `--global` |
|---|---|---|
| `claude` | `.claude/settings.json` | `~/.claude/settings.json` |
| `cursor` | `.cursor/hooks.json` and `.cursor/rules/mast.mdc` | `~/.cursor/hooks.json` only; Cursor has no user-level rules file, and a note says so |
| `vscode` | `.github/hooks/mast.json` | `~/.copilot/hooks/mast.json` |
| `windsurf` | `.windsurf/rules/mast.md`, or `.devin/rules/mast.md` if a `.devin` directory exists | refused (exit 1) |
| `zed` | a marked block in the first existing of `.rules`, `.cursorrules`, `.windsurfrules`, `.clinerules`, `.github/copilot-instructions.md`, `AGENT.md`, `AGENTS.md`, `CLAUDE.md`, `GEMINI.md` | refused (exit 1) |
| `desktop` | nothing | nothing |
```
Options:
--global Write the user-level file instead of the project one
--check Write nothing; exit 0 only if everything is installed and current
--remove Remove what mast installed
--dry-run Print what would be written, and write nothing
```
`claude` and `cursor` get both hooks (`mast hook <harness> session-start` and
`mast hook <harness> search`). **`vscode` gets the session primer only:** VS Code ignores hook
matchers, so a search hook would run before every tool call, and the name of its search tool
is not documented. **`windsurf`, `zed` and `desktop` have no hook system mast can use**, so
the first line of output says that only static instructions are installed. `desktop` writes
nothing: its only channel is the instructions string `mast serve` sends in the MCP handshake,
which needs no setup, and every flag exits 0 after saying so.
The rules files carry the text `mast skill` prints. `mast.mdc` (frontmatter `description` and
`alwaysApply: true`) and the Windsurf `mast.md` (frontmatter `trigger: always_on`) belong to
mast outright, and are overwritten if they differ; there is no marker, the name is the claim.
The Windsurf file stays under Windsurf's documented 12,000-character limit, checked in a test.
`zed` never creates a file: if none of Zed's candidates exists it writes nothing and exits 1,
telling you to create `.rules` and run it again, and otherwise it splices the same marked
block `mast skill --install` uses into the first one that exists. `--remove` takes out that
block, from every candidate that carries it, and leaves the file. The Zed file list is
from memory of Zed's documentation and has not been checked against it.
**Why:** the hook command depends on how mast is installed. A global install writes
`mast hook ...`; a source checkout writes `node "<path to dist/cli/index.js>" hook ...`, a
path specific to your machine, so do not commit that file; a project dependency writes a path
into `node_modules/.bin`. When the package that depends on mast lives in a subdirectory
(`typescript/node_modules`, say), run `setup` with the binary from that directory and the
command points there; `setup` looks beside the `node_modules` it is running out of before
the project root's. A project dependency cannot be installed with `--global`: a
user-level hook pointing into one project's `node_modules` breaks in every other project.
For VS Code the relative `node_modules/.bin/mast` is unverified, because its docs do not
say what directory hooks run in.
For hook files it only ever touches entries whose command ends in `hook <harness> session-start`
or `hook <harness> search`. Every other key, hook and field in the file is kept, in order, and
the file keeps its indentation and trailing newline. Re-running changes nothing and does not
write any file; a changed install (say, source to global) replaces the entry in place. A file
that is not valid JSON, or whose `hooks` have an unexpected shape, is reported and left
untouched (exit 1). `--remove` deletes only mast's entries, and for `vscode` deletes
`mast.json` once nothing else is in it. `--check` cannot be combined with `--remove` or
`--dry-run` (exit 2), nor can an unknown harness be given (exit 2). For `cursor`, `--check`
exits 0 only if both the hooks file and the rules file are current. Nothing here has been
run inside Cursor, VS Code, Windsurf or Zed.
---
### `mast upgrade [path]`
Check for a newer release; print how to install it, and what it will cost.
**Why:** it detects how MAST was installed and prints the matching command rather than
running it, because a CLI cannot reliably distinguish a global install from a dev
dependency. It also reports whether the upgrade bumps the index schema — which forces a
full reindex on the next `serve` — and your package manager cannot tell you that. When
mast's hooks, rules files or skill blocks are installed, it lists the `mast setup <harness>
--check` and `mast skill --install` commands to re-run afterwards.
---
## MCP Tool Reference
MAST registers 13 tools with the MCP server. Every read tool includes a `_stats` block:
```typescript
{
tool: string,
tokens_returned: number,
tokens_full_file_upper_bound: number,
files_referenced: string[],
efficiency_ratio: number, // 1 - (returned / full_file)
duration_ms: number,
}
```
### The signals
Beyond the results, a response carries fields describing what MAST **does not know** about
the answer it just gave. All but the last are omitted entirely when they do not apply, so
their absence carries meaning and their presence is never noise. `truncated` is the one
exception — it is a required field on every `type_context` entry and is present-and-`false`
in the ordinary case, so read it, don't test for it.
| signal | carried by | means |
|---|---|---|
| `stale` | per result of `mast_search`, `mast_implementors` | this result's file changed on disk since it was indexed. The content and line numbers shown may be out of date; no re-parse was attempted (see [JIT Staleness Checks](#jit-staleness-checks)) |
| `file_busy_returning_stale_cache` | `mast_signature`, `mast_callers`, `mast_exports`, `mast_dependencies`, `mast_rename_impact` | a re-parse *was* attempted and lost to a concurrent writer, so the previous chunk was returned. Contended, not wrong by design — retry shortly |
| `index_empty` | the empty answer of any of the eight tools that return a result set | nothing is indexed at all. The answer is empty because there was nothing to answer from, not because nothing matched |
| `unindexed_files` | `mast_search`, `mast_callers`, `mast_implementors`, `mast_rename_impact`, `mast_project_skeleton` (always); `mast_signature`, `mast_exports`, `mast_dependencies` (on an empty answer only) | this many files exist on disk and are not in the index. The answer was computed over an incomplete corpus — an empty or thin result may mean "not indexed" rather than "not present". The tools that claim an exhaustive set report it even when they found something, because a short list reads exactly like a complete one; the three that answer about one named thing report it only when that thing was not found. The count is re-measured in the background when a call finds it older than 30 seconds, and no call waits for it, so it lags by 30 seconds or more: a file created moments ago is not in it yet, and an absent field means "none measured", not "none". After creating files, call `mast_reindex` |
| `pending_edge_repairs` | `mast_callers`, `mast_implementors`, `mast_rename_impact`, with `pending_edge_repairs_hint` | that many indexed files have not had their call and inheritance edges resolved again after another file changed (a background run stopped at its time budget). The answer may miss an entry from one of those files. Run `mast_reindex`, then ask again |
| `results_truncated` | `mast_signature`, `mast_implementors` | the list was capped at `limit`; the field carries the real, uncapped total |
| `exports_truncated` | `mast_exports` | the same, for a module's export list |
| `potential_truncated` | `mast_callers`, `mast_rename_impact` | the unresolved-candidate fetch was capped at 50; the field carries the real match count |
| `truncated` | a `type_context` entry of `mast_signature` | that referenced type's declaration was clipped at 50 lines. The one signal that *is* always present, as a boolean |
Two distinctions that are not flags but carry the same weight: in `mast_callers` and
`mast_rename_impact`, a **`verified_caller`** carries a `resolution` naming how the edge was
statically proven and is safe to act on, while a **`potential_match`** carries a `reason` and
is a name match with no proven edge — review before editing. And an empty result is never
proof of absence: MAST indexes TypeScript, JavaScript, and Markdown only.
---
### `mast_search`
Lexical BM25 + declaration-exact search over the indexed codebase.
```typescript
{
query: string, // natural language or identifier
limit?: number, // max results (default 10, max 50)
language?: "typescript" | "javascript" | "markdown" | null,
file_pattern?: string | null, // glob: "src/api/**"
chunk_type?: "function" | "method" | "class_shell" | "interface" | "type" | "export" | "block" | "doc" | null,
only_exported?: boolean
}
```
**Docs and code share one ranking.** A query that names a declaration is not displaced by Markdown chunks, but a bare word on a docs-heavy repo can be (s01: 60.5% of the top 10 for 12 real queries on mast itself were docs). `language: "typescript"` excludes docs (it also excludes `.js`/`.jsx`, which are `"javascript"`); `language: "markdown"` returns docs only. `language` and `file_pattern` filter inside both rankers before the pool is capped; `chunk_type` and `only_exported` filter after ranking, so `chunk_type: "doc"` can return fewer than `limit` results.
**Returns:** `{ results[], suggestions?, index_empty?, unindexed_files?, _stats }`. Each result includes `file_path`, `start_line`, `end_line`, `content`, `chunk_type`, `symbol_name`, `parent_symbol`, `is_exported`, `match_score` (BM25 score, negative; `null` when the hit came only from ranker D), `rank`, `match_snippet`, an optional `stale` flag, and an optional `related` hint when a method and its class shell both matched (only the higher-ranked one is returned). `suggestions` is present, possibly empty, only when `results` is empty — a zero-result "did you mean" assist. See [The signals](#the-signals) for `stale`, `index_empty`, and `unindexed_files`.
**Why:** `grep` and `glob` find exact strings and require the caller to already know the pattern. `mast_search` ranks by relevance across two signals fused with Reciprocal Rank Fusion:
- **BM25 (FTS5, trigram-tokenized)** — the general-purpose lexical ranker; handles keyword queries and sub-token/camelCase matches.
- **Ranker D (declaration-exact)** — a direct match against a chunk's own `symbol_name` (full name or final dot-segment, case-insensitive). Catches exact-symbol queries BM25's trigram scoring can under-rank. Gated by the `declaration_exact_ranker` config key (default on); when off, `mast_search` is BM25-only.
A chunk both rankers agree on outranks one only one of them found. `file_pattern` and `language` bound the pool **both** rankers draw from, so a scoped search never returns a file outside the scope. `file_pattern` is a glob matched with the same primitive that applies `exclude_patterns` at index time: `*` does not cross `/`, `**` does, `?` is one non-`/` character, matching is case-sensitive, and everything else — `.`, `_`, `-` — is literal.
---
### `mast_project_skeleton`
All exported symbols grouped by file, optionally scoped to a directory.
```typescript
{
directory?: string | null, // path prefix: "src/api"
max_depth?: number, // max subdirectory depth (default unlimited)
file_pattern?: string | null // glob filter on file paths
}
```
**Returns:** `{ files: [{ file_path, exports: string[] }], _stats }`.
**Why:** Before navigating a codebase, an assistant needs orientation — "what exists here?". Reading every file to find its exports is wasteful. `mast_project_skeleton` returns a directory-scoped map of file → exported names in a single call, letting an assistant build a mental model of a subsystem without opening any files.
---
### `mast_exports`
All exported symbols from a single file with type signatures and TSDoc.
```typescript
{
file_path: string // relative to project root
}
```
**Returns:** `{ file_path, exports: [{ name, kind, signature, line, doc }], _stats }`.
**Why:** The natural follow-up to `mast_project_skeleton`. Once an assistant knows which file is relevant, `mast_exports` gives full signatures without the function bodies — enough to understand the public surface of a module without paying for the implementation.
Methods are intentionally omitted (they surface via `mast_signature` on their parent class), so the result stays focused on the module's public contract.
---
### `mast_signature`
Declaration, TSDoc, and resolved parameter type context for a named symbol.
```typescript
{
symbol: string, // e.g. "handleLogin", "AuthService"
file_path?: string | null // narrow to a specific file
}
```
**Returns:** An array of `SignatureResult`, each with `symbol`, `file_path`, `line`, `signature`, `doc`, `params`, `return_type`, and `type_context`.
`type_context` is automatically populated: user-defined PascalCase type names appearing in the signature are resolved to their own signatures via a three-priority lookup — same file first, then named imports, then a global exported-type fallback. Long signatures are truncated at 500 characters. This means a single `mast_signature` call gives the assistant the full type picture for a function without needing separate lookups.
**Why:** When an assistant sees `function processOrder(order: Order, ctx: RequestContext): Promise<Result>`, knowing the signature of `Order`, `RequestContext`, and `Result` is essential for understanding what the function does. Rather than making three more tool calls, `mast_signature` resolves them inline.
---
### `mast_callers`
Who calls a given symbol, split into verified callers (from the symbol graph) and potential matches (from full-text identifier search).
```typescript
{
symbol: string,
file_path?: string | null,
transitive?: boolean, // walk the full call chain (default false)
include_potential?: boolean // include identifier_fts matches (default true)
}
```
**Returns:** `{ verified_callers[], potential_matches[], summary: { verified_count, potential_count, transitive, checker_classified_non_call_site, checker_classified_different_declaration }, _stats }`.
**Why:** Impact analysis before a refactor requires knowing who depends on a symbol. Verified callers are graph-resolved (definitive, no false positives from name collisions). Potential matches are identifier-FTS hits where the call wasn't statically resolvable — they may be false positives but are worth reviewing. Separating the two lets the assistant reason about confidence: if `verified_count` is 3 and `potential_count` is 0, the refactor scope is well-understood. If `potential_count` is 15, there's more uncertainty. Running `mast index --checker` upgrades some potential matches to verified edges (or drops non-call-site noise) — the `checker_classified_*` counts report how many, and are 0 when the checker pass has never run.
**A method called through an interface.** A call on a receiver typed as an interface (`store.get()` with `store: Store`) is a caller of `Store.get`, the method of the interface, and not of the `get` of a class that implements it: the call does not say which class runs. Before changing `MemoryStore.get`, ask for the callers of `Store.get` as well; `mast_implementors Store` gives the classes in the other direction.
---
### `mast_dependencies`
All imports recorded for a file.
```typescript
{
file_path: string
}
```
**Returns:** `{ file_path, imports: [{ module, symbols[], is_external, resolved_path? }], _stats }`.
**Why:** Understanding a file's dependency surface is the first step in reasoning about what it does. External imports (no `resolved_path`) are flagged so the assistant knows the resolution boundary. Internal imports include the resolved path so callers can follow the chain.
---
### `mast_implementors`
All concrete classes that implement a given interface, with their method lists.
```typescript
{
interface_name: string
}
```
**Returns:** `{ results: [{ class_name, file_path, line, methods[] }], _stats }`.
**Why:** In a dependency-injection codebase, `interface_name → implementors` is the answer to "what actually runs here?". Rather than grepping for `implements InterfaceName`, MAST stores explicit `IMPLEMENTS` edges in the graph at index time, making the lookup instantaneous and structurally correct.
---
### `mast_rename_impact`
Composed refactor checklist for renaming a symbol: declaration sites, verified callers, potential matches, and barrel re-exports, in one call.
```typescript
{
symbol: string,
file_path?: string | null
}
```
**Returns:** `{ symbol, declaration_sites[], verified_callers[], potential_matches[], barrel_exports[], summary: { declaration_count, verified_count, potential_count, barrel_count, checklist, checker_classified_non_call_site, checker_classified_different_declaration }, _stats }`.
**Why:** A rename touches more than call sites — barrel re-exports (`export { Foo } from './foo'`, possibly aliased) also need updating, and are easy to miss with a plain caller search. `mast_rename_impact` composes `mast_callers`' machinery with barrel-export detection so the assistant gets one checklist instead of three separate queries.
---
### `mast_outline`
The headings of a Markdown file with each one's line range and size, or one line per `.md` file under a directory (title and status fields), without the text.
```typescript
{
paths: string[], // .md files or directories, 1-20
limit?: number // directory lines (default 50)
}
```
**Returns:** `{ outline, errors?, files_truncated?, _stats }`, where `outline` is plain text: `1-9 75c # Guide` per heading, at every level. A directory lists only the `.md` files the index walks; one with none under it (including one the walk leaves out, such as `node_modules` or a dot directory not in `include_dot_dirs`) is reported in `errors`, and its files can still be named one by one.
**Why:** On a large Markdown file an agent usually needs one part. Replaying grizzly's recorded whole-file reads (spike s03), outline + one section was no larger in 66 of 73 reads of files of 6,000 characters or more, and larger in 10 of 16 under 3,000 — so the tool description tells agents to use it only above about 6,000. The replay assumes one section was needed after the outline.
---
### `mast_section`
Parts of Markdown files, several per call, by heading (`"Install"` or `"Guide > Install"`, subsections included) or by exact line range.
```typescript
{
sections: Array<{
file_path: string,
heading?: string, // or both line fields
start_line?: number,
end_line?: number
}> // 1-20
}
```
**Returns:** `{ sections, errors?, _stats }`. A heading that names more than one section is not guessed at: its `errors` entry lists the candidates.
---
### `mast_reindex`
Trigger a synchronous reindex from within an MCP session.
```typescript
{
full?: boolean // force full reindex (default: incremental)
}
```
**Returns:** `{ files_indexed, files_skipped, chunks_added, chunks_removed, parse_errors, write_errors, duration_ms }`.
**Why:** Long-running editing sessions accumulate staleness — new symbols and files won't be found by `mast_search` until they're indexed (JIT staleness handling keeps *already-indexed* files' line coordinates correct on read, but can't discover a brand-new file or symbol). `mast_reindex` lets the assistant refresh the index on demand — for example, after a large refactor — without leaving the MCP session. The `full` flag is available when incremental state is suspected to be corrupt.
---
### `mast_status`
Health snapshot of the index.
```typescript
// no inputs
```
**Returns:** `{ state_dir, schema_version, index_schema_version, last_indexed, indexed_files, chunk_count, stale_files, stale_breakdown, stale_paths, parse_errors, write_errors, pending_edge_repairs, index_fresh, freshness_cause, seed_commit?, writer }`. `stale_breakdown` splits `stale_files` into `{ changed, unindexed, deleted }`, and `stale_paths` names the files behind each count, up to 20 per category. `pending_edge_repairs` counts indexed files whose edges wait to be resolved again after another file changed; it is not part of `stale_files`, and while it is above zero `mast_callers`, `mast_implementors` and `mast_rename_impact` may miss an edge from one of those files. `writer` is the server that holds the writer lease for this state directory (`{ pid, started_at, mast_version, schema_version, is_this_server }`), or `null` when none is recorded.
`index_fresh` is `true` only when the index has been run at least once, `index_schema_version` (the version that built the index) equals `schema_version` (this binary's), `stale_files = 0` and `pending_edge_repairs = 0`. `freshness_cause` is `null` when fresh, and otherwise one of:
| `freshness_cause` | Meaning |
| --- | --- |
| `"index_version"` | `index.json` names another schema version. Decided before any count. |
| `"stamp_unreadable"` | `index.json` is there and cannot be read, so the version that built the index is not known. |
| `"root_mismatch"` | Most files on disk are not in the index and most indexed files are not on disk: the index was built for another project root. |
| `"phase1_stale"` | Indexed files changed since they were indexed. |
| `"unindexed_files"` | Files on disk that the index has never seen. |
| `"deleted_files"` | Files the index still lists that are gone from disk. |
| `"edge_repair_pending"` | Every file is current, and only `pending_edge_repairs` is above zero. |
When several apply, the first in this order is given; `stale_breakdown` carries the exact split. The tools that read the index return an error over an index of another version; `mast_reindex` rebuilds an older one. `stale_files` counts changed files, files on disk that are not in the index at all, and indexed files that are gone from disk — the same number `mast status` reports, from the same producer.
**Why:** Before a long agentic workflow that depends on accurate code navigation, an assistant can call `mast_status` to confirm the index is fresh, or surface the number of stale files to the user if not.
---
### `mast_efficiency`
Token savings report for the current session or all time.
```typescript
{
scope: "session" | "global",
since_minutes?: number // global scope: restrict to last N minutes
}
```
**Returns:** `{ scope, window_started_at, tokens_returned, tokens_full_file_upper_bound, efficiency_ratio, calls_total, calls_by_tool, tokenizer, counterfactual }`.
The `counterfactual` field is a human-readable sentence: *"Would have cost ~14,200 tokens with naive full-file reads; saved ~11,400 tokens (80.3%)."*
**Why:** Token efficiency is the whole reason MAST exists, but without measurement it is just a claim. Every tool call records tokens returned to `metrics` asynchronously (fire-and-forget, < 1 ms). `mast_efficiency` aggregates those records so the value of precise code navigation is concrete and auditable.
---
## Configuration
MAST reads configuration from `mast.config.json` in the project root, environment variables, or CLI flags. For `state_dir` the order is (highest to lowest): `--state-dir` → `MAST_STATE_DIR` env var → `mast.config.json` → `.mast`. For every other key it is: a `mast init` flag (`--extensions`, `--exclude`) → `mast.config.json` → the saved `<state_dir>/config.json` → built-in defaults. The saved file supplies any key `mast.config.json` does not set (see [`mast walk`](#mast-walk-path)).
| Key | Default | Description |
| -------------------------- | --------------------------------------------------------------------------- | ----------------------------------------------------------------------------|
| `state_dir` | `.mast` | Directory for all index state (relative to project root) |
| `file_extensions` | `.ts,.tsx,.js,.jsx,.md` | Source file extensions to index |
| `exclude_patterns` | `**/node_modules/**`, `**/dist/**`, `**/build/**`, `**/coverage/**`, `**/.next/**`, `**/.turbo/**`, `.mast/**`, `**/*.test.ts`, `**/*.spec.ts` | Glob patterns to skip. A list you set replaces this one, so copy the entries you still want into it. |
| `include_dot_dirs` | `[]` | Dot directories to walk, as paths relative to the project root. Dot directories are skipped unless named here; see [Using MAST in a monorepo](#using-mast-in-a-monorepo). |
| `rrf_k` | `60` | Reciprocal Rank Fusion constant (higher = flatter ranking) |
| `declaration_exact_ranker` | `true` | Fuse ranker D (declaration-exact match) into `mast_search`. Set `false` to restore BM25-only ranking without a code change. |
| `chunk_split_threshold` | `100` | Lines above which a declaration is split into overlapping sub-chunks |
| `context_lines` | `3` | Source lines before/after AST boundaries included in stored content |
| `markdown_heading_depth` | `2` | Maximum ATX heading level (`##`) that starts a new markdown doc chunk |
**`mast.config.json` example:**
```json
{
"state_dir": ".mast",
"exclude_patterns": ["**/node_modules/**", "**/dist/**", "**/build/**", ".mast/**", "**/*.test.ts", "**/fixtures/**"],
"declaration_exact_ranker": true,
"context_lines": 5
}
```
**`MAST_STATE_DIR`** — override the state directory without touching `mast.config.json`. Useful in CI or Docker environments where the project root is read-only.
---
## How It Works
### Indexing
`runIndex` walks the project with `fast-glob`, computes an mtime-based manifest, and diffs it against the stored manifest to find stale, added, and deleted files. For each file that needs processing:
1. **Parse** — `tree-sitter` parses the file into a concrete syntax tree. The TypeScript grammar is used for `.ts` and `.tsx`; the JavaScript grammar for `.js` and `.jsx`. Markdown files are chunked by heading (`markdown_heading_depth`), not parsed with tree-sitter.
2. **Chunk** — the extractor decomposes the CST into typed chunks: `function`, `class_shell` (the class declaration plus member signatures, without bodies), `method` (individual methods), `interface`, `type`, `export`, `block`, and `doc` (markdown sections). Classes are always decomposed so that a search for a single method doesn't return the entire class body.
3. **Sub-chunk** — declarations longer than `chunk_split_threshold` lines are split into overlapping segments so no single chunk is too large to be a useful, self-contained search result.
4. **Symbol graph** — symbols, imports, and edges (IMPLEMENTS, PARENT_OF, POTENTIAL_CALL) are written to SQLite. The two-pass write strategy (all files first, then edges) ensures edges can reference symbols that may be defined in a file parsed later in the same run.
5. **FTS** — chunk content is written to an FTS5 virtual table with a trigram tokeniser, enabling sub-token and camelCase searches. An `identifier_fts` table with a unicode61 tokeniser handles exact-identifier lookups for `mast_callers` potential matches.
Indexing is a single phase — chunk/graph/FTS all update together in one `runIndex` pass; there is no separate embedding step.
### Ranked Search (BM25 + Ranker D via RRF)
A query goes through two rankers:
**BM25 (FTS5):** The query is matched against `chunk_fts` using SQLite's built-in BM25 ranking, over a trigram tokeniser. File-pattern and language filters are pushed into this query as SQL predicates against the `files` table (not as FTS MATCH predicates, because SQLite FTS5 LIKE on UNINDEXED columns is unreliable with MATCH). BM25 scores in SQLite's convention are negative — more negative is a stronger match; `mast_search`'s `match_score` preserves that sign.
**Ranker D (declaration-exact):** A direct SQL predicate against `chunks.symbol_name` — full-name match or final-dot-segment match, case-insensitive, deterministically ordered. Gated by the `declaration_exact_ranker` config key (default on).
**RRF fusion:** The two ranked lists are combined using Reciprocal Rank Fusion:
```
score(chunk) = Σ 1 / (k + rank(chunk))
```
with default `k = 60`. A chunk appearing at rank 1 in both lists scores twice as high as a chunk appearing in only one. Chunks appearing in only one list still score well — neither signal dominates.
### JIT Staleness Checks
Read tools handle a file that changed since it was indexed in one of **two** ways. Which one a tool
uses is fixed per tool, not decided at runtime:
**Re-parse inline** — `mast_signature`, `mast_callers`, `mast_exports`, `mast_dependencies`,
`mast_rename_impact`. Before returning, each calls `jitRefreshFile`, which:
1. Reads the stored mtime for the file from the `files` table. **If there is no row, it returns
immediately** — an unindexed file has no stored mtime to compare against, which is the whole of
why JIT cannot discover new files.
2. Calls `stat()` on the file on disk.
3. If the disk mtime is newer, acquires the `structure.lock` and re-parses the file immediately,
on the request path — the answer waits for it.
If the lock is held by a concurrent writer the previous chunk is returned with
`file_busy_returning_stale_cache` set, so the caller is told rather than quietly served stale data.
**Stat and flag** — `mast_search`, `mast_implementors`. These call `findStaleFiles`, which stats the
files behind the results and sets `stale: true` on the affected ones. **No re-parse is attempted**:
taking a write lock on a ranked result set would serialise the cheapest tools in the package behind
the most expensive operation in it. The line coordinates on a flagged result may be off; the caller
is expected to re-read or call a re-parsing tool.
**Neither** — `mast_project_skeleton` (documented exempt) and `mast_efficiency`.
Both mechanisms only ever touch files the index **already knows**. Neither can discover a file that
was never indexed, so a file created during a session is invisible to every read tool until
something reindexes it — which is why `mast serve` watches by default, and why every
primary-result tool reports `unindexed_files` when the index is nonetheless behind.
### Startup Ladder
`mast serve` starts accepting MCP connections in under 1 second via a four-step ladder:
```
Step 1 Bootstrap state directory; copy Docker seed layer if present;
best-effort remove orphaned pre-vector-store state < 500ms
Step 2 Schema version check; open SQLite < 1s
Step 3 Register all 11 MCP tools; open stdio transport < 500ms
Step 4 Background incremental reindex async
```
All tools are ready to serve as soon as Step 3 completes — there is no reduced-capability startup window. When a pre-built seed index is available at `/opt/mast-seed`, it is copied to the state directory in Step 1 — the background reindex in Step 4 then only needs to process files changed since the seed was built.
### Concurrency Model
One advisory lock coordinates concurrent writers:
- **`structure.lock`** — held by `runIndex` and JIT re-parses. Prevents two writers from modifying the SQLite graph simultaneously.
The lock uses `proper-lockfile` (POSIX advisory locks via a `.lock` marker file). A 10-second stale lock timeout prevents a crashed process from blocking the system indefinitely. Read tools never acquire a write lock — they may see a briefly inconsistent state during a concurrent reindex, and return `file_busy_returning_stale_cache: true` in that case.
### Storage Layout
```
.mast/
graph.db SQLite — symbols, edges, imports, chunks, FTS5 tables, metrics
file_manifest.json mtime snapshot from the last index run
index.json schema version, file count, chunk count, last_indexed
config.json resolved config written at init/serve time
structure lock marker (proper-lockfile target)
```
---
## Token Efficiency
Every tool call records its token count to `metrics` asynchronously. The record includes:
- `tokens_returned` — actual tokens in the response (Anthropic CL100k tokenizer)
- `tokens_full_file_upper_bound` — what a naive full-file read would have cost (when calculable)
- `duration_ms`, `session_id`, and `status`
`metrics_daily` rolls these up by `(day, tool_name)` with a running average for duration and running totals for token counts. The rollup upsert uses an incremental average formula to avoid storing all raw rows indefinitely:
```sql
avg_duration_ms = (old_avg * old_n + new_val) / (old_n + 1)
```
Use `mast metrics --since 7d` for a human-readable table, or `mast_efficiency` from within an MCP session for a machine-readable JSON summary with a `counterfactual` narrative.
---
## History
MAST originally fused BM25 with a vector-embedding search leg (LanceDB + a local ONNX embedding model). Measurement did not support keeping it: the vector store was removed 2026-08-06 per the M2 decision (see [ADR 003](adr/003-2026-08-04-vector-store-deletion.md)). The pre-deletion system — including the embedding pipeline and the eval instruments that measured it — is preserved at the git tag `mast-pre-vector-delete` for anyone re-running that evidence.
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessNo issues