🔎 lens

Token-efficient code & doc retrieval for agents.

The biggest token sink for a coding agent is reading whole files to find a few relevant lines. lens fixes that: index a repo once, then search for ranked snippets, get a symbol outline of a file, or do a surgical line read — pulling just enough context instead of the whole file.

Part of tools-for-agents. Zero dependencies — Node standard library + built-in node:sqlite with FTS5 (BM25 ranking).

Why

Related MCP server: PAMPA

CLI

node src/cli.js index .                       # build the index (incremental on re-run)
node src/cli.js search "websocket reconnect" -k 6 --tokens 1500 --glob 'src/*'
node src/cli.js outline src/server.js         # symbol map, no full read
node src/cli.js read src/server.js 40 80      # surgical line range
node src/cli.js stats                         # index stats

Index location is ./.lens/index.db (override with LENS_DB).

MCP server (for agents)

{
  "mcpServers": {
    "lens": { "command": "node", "args": ["/abs/path/to/lens/mcp/mcp-server.js"],
              "env": { "LENS_DB": "/abs/path/to/repo/.lens/index.db" } }
  }
}

Tools

How it works

• Walks a tree (skipping node_modules, .git, build dirs, binaries, huge files).

• Chunks each file into overlapping line windows and stores them in an FTS5 virtual table.

• search runs an FTS5 MATCH ranked by bm25, then fills results up to a token budget (≈4 chars/token).

• outline is regex-based per language (js/ts, py, go, rust, java, ruby, sql, markdown…).

• index is incremental — files unchanged since last index (by mtime) are skipped.

MIT licensed.