VectorMCP — central cross-repo code index over MCP

Gives Claude Code one central, always-warm index over every Ikshana repo. State a task, get back the repos responsible for it, then search and read their code without cloning anything.

The constraint this is built around

Claude Code clones a fresh copy of a repo for each task and throws it away. So index identity is git object identity, never filesystem location: a chunk's Qdrant point id is uuid5(NS, "{repo_id}:{blob_sha}:{chunk_idx}"). Blob shas are identical in every clone, forever, which means:

• a fresh clone at any commit needs zero re-indexing;

• indexing runs server-side against bare mirrors, never in the agent's checkout;

• the agent only ever contributes deltas for its uncommitted edits, into a disposable scope=session:<id> overlay.

Measured on this codebase: a single commit changes 0.5–0.8% of indexable files, so an incremental pass re-embeds one or two files rather than 927.

Layout

Running it

cp .env.example .env          # then set OPENROUTER_API_KEY
docker compose up -d qdrant postgres
docker compose build indexer

Ports are deliberately non-default — 6335/6336 for Qdrant and 5433 for Postgres — because the Qdrant on 6333 holds live person_reid and face collections and this is a tool whose own collections get dropped and rebuilt.

The vector dimension is fixed at collection creation and OpenRouter does not document it per model, so it is measured:

python3 scripts/probe_embedding.py --rerank   # prints EMBED_DIM -> put it in .env
docker compose run --rm indexer python -m codeindex.cli setup

Bootstrap from the checkouts already on this machine (no Bitbucket credentials needed — LOCAL_REPO_ROOT is bind-mounted read-only at /repos):

docker compose run --rm indexer python -m codeindex.cli add-repo /repos/BACKEND
docker compose run --rm indexer python -m codeindex.cli sync-manifests

Check the ignore policy against real repos before spending anything on embeddings — this needs no Docker, no Postgres and no API key:

python3 scripts/scan_report.py ~/Documents/Ikshana-V2/*/

Using it from Claude Code

claude mcp add --transport http codeindex http://localhost:8080/mcp

Tools, in the order an agent should reach for them:

The UI

http://localhost:8090 — three views over the same data the MCP tools return, so what you see is what the agent sees.

The Index tab also shows the structural graph — extraction totals and the cross-repo coupling table. Open session overlays appear below it, and entering a session id on the Search tab includes that session's uncommitted edits — flagged uncommitted in the results. Repo point counts are scoped to main, so an overlay never inflates a repo's committed total.

Clicking any path:line-line opens the real file at that commit, read from git objects with 25 lines of context either side and absolute line numbers — so a reported range can be checked against the actual file rather than trusted.

Views are linkable: ?q=<query> runs a search (add &sess=<id> to include a session's edits), ?t=<task> routes a task, ?tab=index|search|route switches.

This is served by the stack rather than published anywhere, because it has to reach Qdrant, Postgres and OpenRouter on localhost.

Checks

docker compose run --rm indexer python -m codeindex.cli verify-contents  # reject blobs the bytes disqualify
docker compose run --rm indexer python -m codeindex.cli index            # safe to re-run; unchanged blobs hit the cache
docker compose run --rm indexer python -m codeindex.cli extract          # structural facts (~6s, pure function of the commit)
docker compose run --rm indexer python -m codeindex.cli edges            # derive cross-repo coupling
docker compose run --rm indexer python -m codeindex.cli verify-index     # reconcile manifest -> chunks -> points
docker compose run --rm indexer python scripts/eval_routing.py           # routing regression suite (9 hand-verified cases)
docker compose run --rm indexer python scripts/eval_session.py           # session overlay lifecycle (5 assertions)

verify-index exits non-zero if any repo is short of points, and reconciles against repo_commit.chunk_count rather than the blob_chunk cache -- that cache is content-addressed and shared across repos and commits, so it accumulates and cannot serve as a baseline.

The structural graph

Vector search answers "what looks like this". It cannot answer "what else breaks if I change this", because that is a question about structure — and structure is deterministic, so tree-sitter gives it up for free with no model and no ambiguity. Extraction over all seven repos takes ~6 seconds:

Calls are matched by name, not resolved to a definition. Resolution needs type inference; a name match over 800 Python files is cheap, has no failure modes, and answers the two questions that matter: who calls this, and which other repo is coupled to it. Model.objects.filter(...) also records the root of the dotted chain, so referencing a shared model counts as a usage rather than being invisible behind filter.

Coupling rules

Chosen for what is genuinely invisible to a reader — an import graph misses all of these:

Real edges this found: intozi-ai-suite-backend-dev → docker-mangement via /docker_event_summary_start, cloud-ikshana → intozi-ai-suite-backend-dev via get_intozi_ai_recent_event_data/, and IntoziAiModelArchitectureMaster shared between the suite and ml-ops backends.

route_task reports coupling filtered to the task: an edge is only listed if its evidence appears in the code the search actually surfaced. Every repo here is coupled to the largest one somehow, so listing a repo's couplings wholesale says nothing about the task at hand. Counts read 2 here of 9 — matched versus total — rather than overstating.

Session overlays

The server indexes commits, which is everything except the file the agent is currently editing. So the agent sends its dirty files to sync_workspace and they land in the same collection under scope=session:<id>:

sync_workspace(session_id="fix-alerts", repo="cloud-ikshana",
               files={"path.py": "<contents>"}, deleted=["gone.py"])
search_code(query="...", session_id="fix-alerts")   # sees the edits
end_session("fix-alerts")

Shadowing happens in the Qdrant filter, not after retrieval: paths the session has touched are excluded at main scope. Post-filtering cannot do this — a deleted file has no session points to shadow with, so it would keep returning its committed version forever.

Cost is proportional to what the agent touched. The other 5,400 chunks are already embedded and stay that way, which is the entire point: nothing an agent does can corrupt or re-cost the committed index. Overlays are dropped by end_session and swept after 24 idle hours by the indexer loop, so a crashed agent leaks nothing permanent.

Routing

route_task scores each repo on four signals and returns the ones that survive a relative floor:

Reranking happens before grouping, because RRF fusion scores are rank-based and not comparable across repos. The candidate pool is capped per repo, since one repo holds 78% of all chunks and would otherwise crowd the alternatives out of the reranker entirely. A final substance factor demotes repos whose evidence is all fixtures and docs -- as a threshold, not a ratio, so a small repo whose README genuinely explains its behaviour is not punished for that.

Retrieval design

Small corpus (~26k chunks), so quality wins over every efficiency trade:

• no quantization, and exact=True while under 250k points — brute force with perfect recall instead of an HNSW approximation;

• two dense vectors per point: the code itself, and an LLM-written description of it. Task text is natural language and matches the latter far better than it matches Django internals;

• a sparse lex vector with server-side IDF, because tasks name exact identifiers that dense vectors miss;

• cross-encoder rerank of the fused top ~150 down to ~20. Largest single precision gain in the pipeline.

Status

Phases 0-4 done and verified end to end. 5,467 chunks across 7 repos, 100% with an LLM summary, verify-index clean, routing 9/9 at rank 1, session overlays 5/5, and a structural graph of 6,253 symbols / 52,911 calls / 476 routes with 90 cross-repo coupling edges. Measured: hybrid fusion ~900 ms (including the query embedding round trip), cross-encoder rerank ~1.8 s, extraction ~6 s for everything.

Next: real Bitbucket mirrors, so the index stops depending on whichever branch happens to be checked out locally.

Credential material is excluded, and there is some to deal with

Indexing a secret copies it into a vector store and then into agent context on every loosely-related search, so credential material is rejected on the path where possible (serviceAccountKey.json, .env, *.pem, *.key) and on the bytes where not. Source files are handled at chunk granularity instead -- dropping all of settings.py or 25 chunks of backup_manager.py would cost real code, so the file is indexed and only the offending chunk is held back. Verified: 0 of 5,467 points contain credential markers.

What that turned up in the repos themselves is listed under Known gaps.

Known gaps

• BACKEND is 267/305 files PyArmor-obfuscated -- 12.2 MB of hex-escaped bytecode against 14 KB of real source. looks_generated rejects it. Read intozi-ai-suite-backend-dev instead; it holds the readable form.

• BACKEND is also a shallow single-commit clone, so incremental diffing cannot work there until a real mirror replaces it.

• 34 of 5,512 chunks (0.6%) have no summary after a batch and a singleton retry. They still carry code and lexical vectors, and each index run retries them.

• A Firebase/GCP service account key is committed at intozi_ai_suite_api/intozi_alert_management/mobile_alert_notifications/serviceAccountKey.json, in both BACKEND and intozi-ai-suite-backend-dev.

• A literal AWS access key id (AKIA...) is hardcoded in intozi_ai_suite/settings.py, face_server/config.py and intozi_ai_suite_api/intozi_consumer_management/msk_kafka.py. Excluded from the index; still in git history, which the index cannot fix.

• Cross-repo blob dedup is negligible here (8 blobs, 29 kB). The incremental case is what content addressing actually buys.

• celery_task derives 0 edges: the 25 tasks found are all called within their own repo. The rule is right, there is simply nothing to find yet.

• blob_chunk and embedding_cache are keyed on blob sha without language. No blob in these repos is indexed under two languages, but the same content at two differently-typed paths would collide.