codexlens-search

Semantic code search engine with MCP server for Claude Code.

Hybrid search: vector + FTS + AST graph + ripgrep regex — with RRF fusion and reranking.

中文文档

Quick Start

pip install codexlens-search[all]

Add to your project .mcp.json:

{
  "mcpServers": {
    "codexlens": {
      "command": "uvx",
      "args": ["--from", "codexlens-search[all]", "codexlens-mcp"],
      "env": {
        "CODEXLENS_EMBED_API_URL": "https://api.openai.com/v1",
        "CODEXLENS_EMBED_API_KEY": "${OPENAI_API_KEY}",
        "CODEXLENS_EMBED_API_MODEL": "text-embedding-3-small",
        "CODEXLENS_EMBED_DIM": "1536"
      }
    }
  }
}

That's it. Claude Code will auto-discover the tools: index_project -> Search -> locate.

To enable LLM-enhanced search (locate), add LLM API keys:

{
  "mcpServers": {
    "codexlens": {
      "command": "uvx",
      "args": ["--from", "codexlens-search[all]", "codexlens-mcp"],
      "env": {
        "CODEXLENS_EMBED_API_URL": "https://api.openai.com/v1",
        "CODEXLENS_EMBED_API_KEY": "${OPENAI_API_KEY}",
        "CODEXLENS_EMBED_API_MODEL": "text-embedding-3-small",
        "CODEXLENS_EMBED_DIM": "1536",
        "CODEXLENS_LLM_EXPAND_API_KEY": "${GLM_API_KEY}",
        "CODEXLENS_LLM_EXPAND_MODEL": "glm-5-turbo",
        "CODEXLENS_LLM_EXPAND_API_BASE": "https://open.bigmodel.cn/api/paas/v4/"
      }
    }
  }
}

Install

Choose the install that matches your platform:

# Minimal — CPU inference (fastembed bundles onnxruntime CPU)
pip install codexlens-search

# Windows GPU — DirectML, any DirectX 12 GPU (NVIDIA/AMD/Intel)
pip install codexlens-search[directml]

# Linux/Windows NVIDIA GPU — CUDA (requires CUDA + cuDNN)
pip install codexlens-search[cuda]

# Auto-select — DirectML on Windows, CPU elsewhere
pip install codexlens-search[all]

Platform Recommendations

Note: On Windows, if you install the base package without [directml], the MCP server will auto-detect the missing GPU runtime and install onnxruntime-directml on first launch. GPU takes effect from the second start.

What's Included

All install variants include:

• MCP server — codexlens-mcp command

• AST parsing — tree-sitter symbol extraction + graph search

• USearch — high-performance HNSW ANN backend (default)

• FAISS — ANN + binary index backend (Hamming coarse search)

• File watcher — watchdog auto-indexing

• Gitignore filtering — recursive .gitignore support

• Focused search — when no index exists, greps relevant files, indexes only those (~10s), then runs semantic search — no waiting for full index build

ANN Backend Selection

Three backends for approximate nearest neighbor search, auto-selected in order:

Override with CODEXLENS_ANN_BACKEND:

CODEXLENS_ANN_BACKEND=faiss    # use FAISS (GPU when available)
CODEXLENS_ANN_BACKEND=usearch  # use USearch (default)
CODEXLENS_ANN_BACKEND=hnswlib  # use hnswlib
CODEXLENS_ANN_BACKEND=auto     # auto-select (usearch > faiss > hnswlib)

MCP Tools

Search

Hybrid code search combining semantic vector, FTS, AST graph, and ripgrep regex.

Parameters: project_path, query, mode, scope (restricts auto/regex to subdirectory)

Results capped by CODEXLENS_TOP_K env var (default 10).

Cold Start Search

When no index exists, auto mode uses a focused search pipeline instead of waiting for a full index build:

1. Expand query — split camelCase/snake_case into search terms

2. Grep files — rg --count finds top 50 relevant files, ranked by match count

3. Index — embed only those 50 files (~8-10s with GPU)

4. Search — semantic vector search on the fresh index

5. Background — full index builds asynchronously for next queries

This gives semantic results in ~10s vs ~100s for a full index build.

locate

LLM-enhanced code localization — finds files relevant to a bug fix or feature request.

Uses a single LLM call to expand the query into symbols, concepts, and alternative search queries, then runs multiple pipeline searches and merges results via reciprocal rank fusion (RRF). Achieves recall comparable to iterative agent approaches at ~10s latency (vs ~60s for full agent loops).

Parameters: project_path, query, top_k (default 10), llm_expand (default true)

Set llm_expand=false to use plain pipeline search without LLM.

Requires LLM API configuration (any OpenAI-compatible API — GLM, DeepSeek, Qwen, OpenAI, etc.):

{
  "mcpServers": {
    "codexlens": {
      "command": "uvx",
      "args": ["--from", "codexlens-search[all]", "codexlens-mcp"],
      "env": {
        "CODEXLENS_EMBED_API_URL": "https://api.openai.com/v1",
        "CODEXLENS_EMBED_API_KEY": "${OPENAI_API_KEY}",
        "CODEXLENS_EMBED_API_MODEL": "text-embedding-3-small",
        "CODEXLENS_EMBED_DIM": "1536",
        "CODEXLENS_LLM_EXPAND_API_KEY": "${GLM_API_KEY}",
        "CODEXLENS_LLM_EXPAND_MODEL": "glm-5-turbo",
        "CODEXLENS_LLM_EXPAND_API_BASE": "https://open.bigmodel.cn/api/paas/v4/"
      }
    }
  }
}

Falls back to plain pipeline search if no LLM API key is configured.

How it works

1. LLM Query Expansion — a single LLM call extracts symbols, technical concepts, error terms, and alternative search queries from the issue description

2. Multi-Query Search — runs the original query plus expanded queries through the hybrid pipeline (vector + FTS + graph + regex)

3. RRF Fusion — merges all result lists via reciprocal rank fusion for robust ranking

4. Reranking — applies cross-encoder reranker (if configured) for final top-k

index_project

Build, update, or inspect the search index.

Parameters: project_path, action, scope

find_files

Glob-based file discovery. Parameters: project_path, pattern (default **/*)

Max results controlled by CODEXLENS_FIND_MAX_RESULTS env var (default 100).

watch_project

Manage file watcher for automatic re-indexing on file changes.

Parameters: project_path, action (start / stop / status)

AST Features

Enabled by default. Disable with CODEXLENS_AST_CHUNKING=false.

• Smart chunking — splits at symbol boundaries instead of fixed-size windows

• Symbol extraction — 12 kinds: function, class, method, module, variable, constant, interface, type_alias, enum, struct, trait, property

• Cross-references — import, call, inherit, type_ref edges

• Graph search — seeded from vector/FTS results, BFS expansion with adaptive weights

• Query expansion — two-hop symbol vocabulary expansion for natural language queries

Languages: Python, JavaScript, TypeScript, Go, Java, Rust, C, C++, Ruby, PHP, Scala, Kotlin, Swift, C#, Bash, Lua, Haskell, Elixir, Erlang.

Configuration Examples

Reranker (best quality)

Add reranker API on top of the Quick Start config:

"CODEXLENS_RERANKER_API_URL": "https://api.jina.ai/v1",
"CODEXLENS_RERANKER_API_KEY": "${JINA_API_KEY}",
"CODEXLENS_RERANKER_API_MODEL": "jina-reranker-v2-base-multilingual"

Multi-Endpoint Load Balancing

"CODEXLENS_EMBED_API_ENDPOINTS": "https://api1.example.com/v1|sk-key1|model,https://api2.example.com/v1|sk-key2|model",
"CODEXLENS_EMBED_DIM": "1536"

Format: url|key|model,url|key|model,... — replaces single-endpoint EMBED_API_URL/KEY/MODEL.

Local Models (Offline)

No API needed — fastembed runs the model locally via ONNX runtime.

# List available models
codexlens-search list-models

# Pre-download models (optional — auto-downloads on first use)
codexlens-search download-models

# Download a specific model
codexlens-search download-model nomic-ai/nomic-embed-text-v1.5-Q

{
  "mcpServers": {
    "codexlens": {
      "command": "codexlens-mcp",
      "env": {
        "CODEXLENS_DEVICE": "directml"
      }
    }
  }
}

Default local model: BAAI/bge-small-en-v1.5 (384d, 512 tokens). To use a different model:

{
  "mcpServers": {
    "codexlens": {
      "command": "codexlens-mcp",
      "env": {
        "CODEXLENS_EMBED_MODEL": "nomic-ai/nomic-embed-text-v1.5-Q",
        "CODEXLENS_EMBED_DIM": "768",
        "CODEXLENS_DEVICE": "directml"
      }
    }
  }
}

Available Local Models

General

Code / Long Context

Chinese / Multilingual

CODEXLENS_EMBED_DIM must match the model's output dimension. Mismatched dim will cause indexing errors.

Recommended for Code Search

Manual Model Download

If codexlens-search download-model fails (e.g. behind a firewall), you can download models manually:

1. Find the ONNX repo — fastembed remaps model names. Check the actual HuggingFace repo:

   Model name	Actual HF repo
   BAAI/bge-small-en-v1.5	qdrant/bge-small-en-v1.5-onnx-q
   nomic-ai/nomic-embed-text-v1.5-Q	nomic-ai/nomic-embed-text-v1.5
   jinaai/jina-embeddings-v2-base-code	jinaai/jina-embeddings-v2-base-code

   Run codexlens-search list-models --json to check cache paths.

2. Download from HuggingFace — clone or download the repo:

   # Using git (requires git-lfs)
   git lfs install
   git clone https://huggingface.co/qdrant/bge-small-en-v1.5-onnx-q
   
   # Or download via huggingface-cli
   pip install huggingface-hub
   huggingface-cli download qdrant/bge-small-en-v1.5-onnx-q --local-dir ./model-files

3. Place in cache directory — the cache follows HuggingFace Hub layout:

   <cache_dir>/
     models--<org>--<model>/
       snapshots/
         <commit_hash>/
           model_optimized.onnx   (or model.onnx)
           tokenizer.json
           config.json
           special_tokens_map.json
           tokenizer_config.json

   Default cache locations:

   • Windows: %LOCALAPPDATA%\fastembed_cache or %TEMP%\fastembed_cache

   • Linux/macOS: /tmp/fastembed_cache

   • Custom: set CODEXLENS_MODEL_CACHE_DIR

   Example for BAAI/bge-small-en-v1.5:

   mkdir -p /tmp/fastembed_cache/models--qdrant--bge-small-en-v1.5-onnx-q/snapshots/main/
   cp model-files/*.onnx model-files/*.json \
      /tmp/fastembed_cache/models--qdrant--bge-small-en-v1.5-onnx-q/snapshots/main/

4. Verify — run codexlens-search list-models and check the ● status.

China Mirror

"CODEXLENS_HF_MIRROR": "https://hf-mirror.com"

Custom Model Cache

"CODEXLENS_MODEL_CACHE_DIR": "/path/to/cache"

GPU

Windows: pip install codexlens-search[directml] — works with any DirectX 12 GPU (NVIDIA/AMD/Intel). No CUDA needed. Even without [directml], the server auto-installs it on first launch.

Linux: pip install codexlens-search[cuda] adds CUDA support (requires CUDA + cuDNN).

Auto-detection priority: CUDA > DirectML > CPU

• Embedding — ONNX runtime selects best available GPU provider, ~12x faster than CPU

• FAISS — index auto-transfers to GPU 0 (CUDA only)

Force specific device: CODEXLENS_DEVICE=directml / cuda / cpu

CLI

codexlens-search --db-path .codexlens sync --root ./src
codexlens-search --db-path .codexlens search -q "auth handler" -k 10
codexlens-search --db-path .codexlens status
codexlens-search list-models
codexlens-search download-models
codexlens-search download-model BAAI/bge-base-en-v1.5
codexlens-search delete-model BAAI/bge-small-en-v1.5

Environment Variables

Local Model

Embedding API (overrides local model)

Reranker

LLM Query Expansion (locate tool)

Fallback: If CODEXLENS_LLM_EXPAND_* vars are not set, the system also checks CODEXLENS_AGENT_LLM_* vars for backward compatibility.

Features

MCP Tool Defaults

Tuning

Architecture

Query -> [QueryExpander] -> expanded query (NL queries only)
          |-> [Embedder] -> query vector
          |     |-> [FAISS Binary] -> candidates (Hamming)
          |     |     +-> [USearch/FAISS HNSW] -> ranked IDs (cosine)
          |     +-> [FTS exact + fuzzy] -> text matches
          |-> [GraphSearcher] -> symbol neighbors (seeded from vector/FTS)
          +-> [ripgrep] -> regex matches
               +-> [RRF Fusion] -> merged ranking
                     +-> [Reranker] -> final top-k

Development

git clone https://github.com/catlog22/codexlens-search.git
cd codexlens-search
pip install -e ".[dev,all]"
pytest

License

MIT