layergrep

Local, offline semantic + literal code search. Point it at a project, ask it a question in plain language ("handler that updates user status"), get back implementation locations grouped by architectural layer and module — not a flat ranked list.

Available both as a CLI and as an MCP server, so an LLM client (e.g. Claude Code) can call it directly during a conversation.

Why "layergrep" and not "codesearch"

Plain "search" implies a single list of equivalent results ranked by score. That's not how this tool works: it's a matrix — rows are layers (a cross-cutting architectural role: frontend, backend/api, models, config, ...), columns are business entities/modules (a sibling package, a Rust crate, or a feature that spans both frontend and backend files) — and results are grouped by both, so a large module can't drown out a smaller one just because it has more code. -grep signals "searches code" the way ripgrep/ast-grep/semgrep do; layer is the actual differentiator.

Related MCP server: Cortex

How it works

1. Chunking — AST-based (tree-sitter), not fixed-size windows. Functions, classes/structs, and methods for .py/.js/.jsx/.ts/.tsx/.rs; adjacent top-level statements (constants, route registration blocks) get grouped too, so plain data/config files aren't invisible to search. JSON files (e.g. translation catalogs) get both an exact literal lookup and adaptive size-budgeted chunking for semantic search.

2. Embedding — each chunk is embedded with a configurable sentence-transformers model (default intfloat/multilingual-e5-small; also supports bge-small/bge-m3/e5-large/the Qwen3-Embedding family). Stored in SQLite via sqlite-vec — no external vector database.

3. Retrieval — hybrid: literal grep on exact identifiers/paths first, then per-(layer, module) vector search (top-k within each bucket, not overall — the reason results don't collapse into a single dominant category), then two structural expansion passes: the import graph (handler → model/config/constants it imports) and shared string literals across layers (the only non-embedding bridge between languages, e.g. a route path used by both a JS frontend and a Python backend).

4. Everything is project-specific and configurable, not hardcoded — layer rules, translation file paths, module depth, and noise-filtering thresholds all live in .layergrep.json (see below).

Install

Requires Python 3.12+. Heavy ML dependencies (torch, sentence-transformers, tree-sitter grammars) — expect a real download/install on first setup, this is an embedding-model-backed tool, not a lightweight script.

git clone <this-repo>
cd layergrep
pip install -e .

Or, once published, with uv — no manual venv needed:

uvx --from git+<this-repo-url> layergrep-mcp-server

Quickstart (CLI)

# 1. Draft a config from the target project's own structure (review it — it's a draft)
layergrep init-config --root /path/to/project

# 2. First time on this machine: download the embedding model (needs network, one-time)
layergrep install-model

# 3. Build the index (incremental — safe to re-run after code changes)
layergrep index --root /path/to/project

# 4. Search
layergrep "handler that updates user status" --root /path/to/project

# Optional: check whether the default noise-filtering thresholds fit this project's corpus size
layergrep calibrate-thresholds --root /path/to/project

layergrep "index" (the literal word) needs the -- escape hatch, since index alone is a subcommand: layergrep -- "index".

Quickstart (MCP server)

Add to .mcp.json in the target project:

{
  "mcpServers": {
    "layergrep": {
      "command": "/path/to/layergrep/.venv/Scripts/python.exe",
      "args": ["/path/to/layergrep/mcp_server.py"],
      "env": {
        "LAYERGREP_ROOT": "/path/to/target/project"
      }
    }
  }
}

(Or, if installed via pip/uv, command can just be layergrep-mcp-server directly — no venv path needed.)

Optional env vars: LAYERGREP_SUBDIR (index only a subtree of the project, default: the whole root) and LAYERGREP_MODEL (override the embedding model). Three tools are exposed:

• layergrep(query) — the search itself, returns markdown grouped by layer/module.

• index_codebase(force=False) — incremental (re)index; call after code changes.

• install_model() — one-time model download if the server reports it isn't cached yet (it won't crash on a fresh machine — it tells you to call this instead).

Configuration: .layergrep.json

Optional, lives at the target project's root (not inside .layergrep/, which holds only generated artifacts — the index DB and log — and should be gitignored). Missing entirely → everything falls into one default_layer, which is a valid, honest result for a project that hasn't been configured yet — never silently reuses another project's layer rules.

{
  "layers": [
    {"name": "frontend", "dirs": ["frontend", "client"], "files": []},
    {"name": "backend/api", "dirs": ["api", "routes"], "files": ["views.py"]},
    {"name": "models", "dirs": ["models"], "files": []},
    {"name": "config", "dirs": ["config"], "files": ["settings.py"]}
  ],
  "default_layer": "backend/other",
  "translations": {"files": ["langs/ru.json"]},
  "extra_excluded_dirs": ["vendor"],
  "forced_add": ["target"],
  "module_depth": 1,
  "modules": [
    {"name": "audio-engine", "dirs": [], "files": ["engine.rs", "Deck.tsx"]}
  ],
  "literal_noise_threshold": 30,
  "import_noise_threshold": 15
}

layergrep init-config drafts this file from common Flask/Django/FastAPI/ Express/React-shaped conventions — review before relying on it, it can't guess a project's own vocabulary. It also detects which languages are present, and for Rust projects reads real crate names straight out of every Cargo.toml under the root (not the directory name — a directory called libxdo-sys-stub can hold a crate actually named libxdo-sys) to suggest module_depth; crates aren't suggested as layers, since a crate is a sibling package (the modules axis), not an architectural role. layergrep calibrate-thresholds reports the real match-count distribution for the literal_noise_threshold/import_noise_threshold fields on an already- indexed project, instead of leaving you to guess whether the defaults (tuned on one ~8000-chunk corpus) fit yours.

Development

pip install -e ".[dev]"
pytest              # full suite
pytest -m "not slow"  # skip tests that load the real embedding model / spawn subprocesses

License

MIT — see LICENSE.