# CodeContext MCP

[![verify](https://github.com/dileepreddy27/codecontext-mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/dileepreddy27/codecontext-mcp/actions/workflows/ci.yml)

**Cited repository context for coding agents, backed by Tree-sitter and a local vector index.**

CodeContext turns Python, Go, and Rust declarations into bounded, searchable source chunks. A Go CLI coordinates parser processes and publishes a LanceDB snapshot; a read-only MCP server exposes search, chunk lookup, and index statistics. Everything runs on a CPU without API keys or model downloads.

![Architecture](docs/architecture.svg)

### Why this exists

Copying entire files into an agent conversation wastes context and obscures the relevant implementation. CodeContext returns selected declarations with relative paths, source line ranges, and stable content IDs. A caller controls how many results and how many **source characters** are returned. This is a context-selection tool; it does not measure or promise tokenizer-specific token savings.

### Implemented architecture

| Layer | Implementation | Engineering choice |
| --- | --- | --- |
| Repository discovery | Go + `git ls-files` | Honors Git ignore rules for untracked files; skips symlinks, binary/invalid UTF-8, common sensitive names, vendor directories, and files over 1 MiB |
| Parallel execution | Bounded Go worker pool | 1–16 Python parser processes, one batch per process; a failed batch prevents snapshot publication |
| Syntax parsing | Tree-sitter native bindings | Python functions/classes, Go functions/methods/types, Rust functions/structs/enums/impls/traits |
| Chunking | AST boundaries + bounded windows | Top-level declarations avoid nested duplication; at most 80 accumulated line parts / 6,000 characters per chunk |
| Embeddings | 256-dimensional signed BLAKE2b token hashing | Deterministic normalized lexical baseline; camelCase and snake_case splitting, no learned semantics |
| Storage/search | LanceDB + Arrow | Versioned full-table replacement; exact cosine nearest-neighbor search, no approximate index |
| Agent interface | Go JSON-RPC over stdio | MCP `2025-11-25` lifecycle and three read-only tools |

**Stack scope:** the application is Go with a Python/native parsing and storage worker. Rust source is supported, and LanceDB uses native internals, but this repository does not contain a custom Rust indexing engine. Python makes packaged Tree-sitter and LanceDB bindings portable. PostgreSQL/pgvector, learned embeddings, and GPU acceleration are not implemented. MCP is the access protocol, **not a vector-storage format**.

### Quick start

Requires **Go 1.24.7+**, **Python 3.12**, and **Git**. Run from the cloned repository. Dependency installation needs internet; indexing and retrieval do not.

```bash
git clone https://github.com/dileepreddy27/codecontext-mcp.git
cd codecontext-mcp
python -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements-lock.txt
go build -o bin/codecontext ./cmd/codecontext
python scripts/demo.py --binary bin/codecontext
bin/codecontext index --root fixtures/demo --db .codecontext --workers 2
bin/codecontext search --db .codecontext --query "evict expired cache entries" --limit 3 --budget 2000
```

PowerShell uses `.venv\Scripts\Activate.ps1` and `go build -o bin/codecontext.exe ./cmd/codecontext`; then run `python scripts/demo.py` and `.\bin\codecontext.exe`. If activation is unavailable, pass `--python .venv/Scripts/python.exe` to each command or set `CODECONTEXT_PYTHON` to that executable.

The demo creates a temporary Git repository from the original synthetic fixtures, builds a real LanceDB index, checks three expected search results, and exercises MCP initialization, tool listing, search, and statistics. It deletes its temporary database afterward. No fixture output is substituted for a real run.

Index your own repository with `codecontext index --root /path/to/repo --db /path/to/index`. Each database holds **one snapshot**: indexing another root replaces it. Refreshing removes stale chunks by rebuilding everything. The source snapshot is limited to 64 MiB and held in memory; choose a smaller subdirectory for larger projects.

### Connect an MCP client

Build the binary and index a repository first. Adapt this standard stdio configuration to your client's settings, using absolute paths for all three locations:

```json
{
  "mcpServers": {
    "codecontext": {
      "command": "/absolute/path/codecontext",
      "args": ["serve", "--db", "/absolute/path/index", "--python", "/absolute/path/.venv/bin/python"]
    }
  }
}
```

| Tool | Arguments | Result |
| --- | --- | --- |
| `search_code` | `query`, optional `limit` (1–20), `budget` (128–32,000) | Ranked chunks, citations, cosine distance, truncation flag |
| `get_chunk` | 24-character hexadecimal `id` | Full indexed chunk, or an empty list for an unknown ID |
| `index_stats` | none | Row count, model ID, dimensions, supported languages, storage/search type |

`budget` bounds the combined returned source text, excluding JSON and metadata. A truncated result retains its original source range; use `get_chunk` for the stored full chunk. IDs change when source or relative location changes. Results are snapshots, not live file reads. The server never executes source code and clients cannot select filesystem paths through tool arguments. Protocol errors use JSON-RPC errors; execution failures use MCP `isError`. stdout contains only protocol JSON in server mode.

### Verification and measurements

```bash
go test -v ./...
go vet ./...
python -m pytest -q
python scripts/demo.py --binary bin/codecontext --output work/demo.json
# Optional repeated-fixture workload, not a real repository benchmark:
python scripts/demo.py --binary bin/codecontext --copies 100 --output work/repeated.json
```

CI runs Go tests, static checks, Python tests, and the real CLI/MCP demo on Linux and Windows; Linux also runs the Go race detector. Each job uploads its actual demo report. See [verification evidence](docs/verification.md) for measured results and the boundary between checks run and work not validated.

The tests exercise all three grammars, malformed syntax reporting, UTF-8 boundaries, stable IDs, bounded chunks, normalization, vector retrieval, snapshot replacement/deletion, empty indexes, input validation, Git ignores, source limits, symlink exclusion, protocol lifecycle, malformed requests, notification suppression, and database-argument rejection. The three demo queries are **handcrafted smoke checks**, not a retrieval-quality benchmark or held-out evaluation dataset.

### Tradeoffs and limits

- No claim of indexing the Linux kernel in minutes. The current implementation targets inspectable small-to-medium snapshots, not colossal codebases.
- Each storage/search call starts Python and imports LanceDB. This simplifies isolation and packaging but adds noticeable latency; a persistent worker is future work.
- Hash collisions, lexical mismatch, and lack of learned semantics limit retrieval quality. Exact search scans vectors; there is no ANN index or relevance threshold.
- Imports and loose top-level code are omitted when declarations exist. Classes and Rust impls stay whole before windowing; nested methods are not independently indexed. Large/minified declarations may be split inside syntax, and inserted line breaks in long single-line windows mean stored text is a readable fragment rather than an exact byte slice.
- Syntax-error files are indexed with `parse_error: true`; this does not validate code. Only `.py`, `.go`, and `.rs` are supported.
- Full refresh only: no file watcher, incremental cache, cross-repository federation, or pgvector adapter. Memory use includes source, chunks, JSON copies, and vectors.
- Git ignores do not exclude tracked files. Name-based sensitive-file exclusions are not secret detection. Inspect the repository before indexing and protect the database as source code; old LanceDB versions may retain prior data until cleaned up.
- Filesystem checks are best-effort for trusted local repositories, not a hostile concurrent filesystem sandbox. Do not mutate source paths during indexing. Index lock files prevent ordinary competing CLI writers; after a crash, remove a stale lock only when no writer remains.
- MCP support is the documented stdio tool subset for `2025-11-25`, not full protocol certification. No HTTP transport, authentication, resources, prompts, or remote service. Actual third-party desktop client integration is not claimed.
- Indexed comments can contain prompt injection. Clients must treat all retrieved code as untrusted data, never higher-priority instructions.

### Project materials

- [Architecture and design decisions](docs/design.md)
- [Verified results and limitations](docs/verification.md)
- [Evidence-backed resume bullets](docs/resume.md)
- [Dependency and fixture licensing](THIRD_PARTY.md)

Original code and synthetic fixtures are [MIT licensed](LICENSE).