Semantic Code Intelligence

Local-first semantic search and cited code walkthroughs for software repositories.

Semantic Code Intelligence parses a repository into symbol-aware chunks, indexes those chunks with FAISS and BM25, fuses both result sets, and reranks the strongest candidates with a cross-encoder. Results include exact file paths and line ranges. Everything runs locally; no cloud API key is required.

What it provides

• Hybrid semantic and lexical code search

• Exact symbol, path, and contextual-term boosting

• Search reliability labels based on retrieval agreement

• Python AST parsing and structural parsing for common programming languages

• Exact citations such as src/auth.py:L42-L67

• Browser dashboard and REST API

• CLI, MCP, and LSP interfaces

• Local Ollama-powered code walkthroughs with a deterministic evidence fallback

• FAISS, BM25, and SQLite index persistence

• Incremental filesystem watching

• Symbol and dependency graphs

• Reproducible indexing and retrieval benchmarks

Related MCP server: Qurio MCP Server

Requirements

• macOS or Linux

• Python 3.10 or newer

• Git

• Approximately 2–4 GB of free disk space for Python dependencies and local model caches

• Optional: uv for faster environment management

• Optional: Ollama for generated code walkthroughs

The first indexing and reranking operations require internet access to download Hugging Face model weights. After the models are cached, retrieval works offline.

Quick start from a clean machine

1. Clone the repository

git clone https://github.com/saitarrun/Semantic-code-intelligence.git
cd semantic-code-intelligence

2. Create an environment and install the application

Using uv:

uv venv
source .venv/bin/activate
uv pip install -e .

Using standard Python tooling:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e .

Windows is not currently a tested target, but the equivalent activation command is .venv\Scripts\activate.

3. Download the retrieval models and create an index

Model downloads are deliberately disabled by default so normal application requests never trigger unexpected network traffic. Explicitly enable downloads during the first index and query:

export CODE_INTEL_ALLOW_MODEL_DOWNLOADS=1

code-intel index .
code-intel query "Where is HybridRetrievalPipeline implemented?" --citations-only

unset CODE_INTEL_ALLOW_MODEL_DOWNLOADS

This prepares:

• sentence-transformers/all-MiniLM-L6-v2 for dense embeddings

• cross-encoder/ms-marco-MiniLM-L-6-v2 for reranking

The repository index is stored in .code_intel_index/. The directory contains the FAISS index, BM25 data, and SQLite metadata and should not be committed.

4. Start the web application

code-intel serve --host 127.0.0.1 --port 8000

Open http://127.0.0.1:8000.

The dashboard includes:

• Semantic Search

• Code Walkthrough

• Dependency Map

• Diff and LSP tools

• Repository selection and reindexing controls

• Per-stage latency and retrieval-reliability indicators

Index another repository

Index data is stored inside the target repository by default:

code-intel index /absolute/path/to/project

Search that repository:

code-intel query \
  "How are access tokens validated?" \
  --dir /absolute/path/to/project

Use a separate index directory when the source repository should remain untouched:

code-intel index /absolute/path/to/project \
  --index-dir /absolute/path/to/index-storage

code-intel query \
  "Where is the database connection pool created?" \
  --dir /absolute/path/to/project \
  --index-dir /absolute/path/to/index-storage

Force a clean rebuild after changing parser or embedding behavior:

code-intel index /absolute/path/to/project --force

Semantic search

Hybrid mode is recommended. It combines natural-language similarity with exact identifier matching:

code-intel query "How does the application serve the web UI?"

Exact symbol search:

code-intel query "Where is serve_ui implemented?"

Return more results:

code-intel query "authentication middleware" --top-k 10

Show citations without printing code:

code-intel query "database transaction rollback" --citations-only

Select an individual retrieval strategy for diagnostics:

code-intel query "PaymentProcessor" --mode sparse
code-intel query "logic responsible for charging a customer" --mode dense
code-intel query "charge customer payment" --mode hybrid

Disable cross-encoder reranking when lower latency matters more than precision:

code-intel query "configuration loader" --no-rerank

How ranking works

The default hybrid pipeline performs these stages:

1. Expand common developer intents with deterministic code-domain terms.

2. Retrieve up to 50 dense FAISS candidates.

3. Retrieve up to 50 lexical BM25 candidates.

4. Fuse up to 60 unique candidates with Reciprocal Rank Fusion.

5. Rerank up to 40 candidates with a local cross-encoder.

6. Boost exact symbols, paths, and contextual term matches.

7. Remove duplicate citations and limit repetitive same-file results.

8. Return a reliability label with the evidence behind it.

Reliability is not an LLM confidence score. It reports observable retrieval signals such as dense/lexical agreement, exact symbol matches, path overlap, and semantic similarity.

Code walkthroughs

Deterministic evidence mode

This mode does not require Ollama. It returns retrieved symbols, scopes, dependencies, source blocks, and citations without inventing behavior:

code-intel ask \
  "How does the indexing pipeline persist metadata?" \
  --provider extractive

Generated local walkthroughs with Ollama

Install and start Ollama, then download the default model:

ollama pull qwen2.5-coder:7b

Run a cited walkthrough:

code-intel ask "Explain the hybrid retrieval control flow"

Use another local model or Ollama server:

export CODE_INTEL_OLLAMA_MODEL=deepseek-coder-v2:lite
export OLLAMA_BASE_URL=http://127.0.0.1:11434

If Ollama cannot be reached, the application clearly labels the response extractive-fallback and returns deterministic source evidence.

Interactive CLI

Start a continuous search session:

code-intel interactive --dir /absolute/path/to/project

Inspect index statistics:

code-intel stats --dir /absolute/path/to/project

Display all commands:

code-intel --help
code-intel query --help

REST API

Start the server:

code-intel serve --host 127.0.0.1 --port 8000

Health check:

curl http://127.0.0.1:8000/api/health

Index a repository:

curl -X POST http://127.0.0.1:8000/api/index \
  -H 'Content-Type: application/json' \
  -d '{
    "target_dir": "/absolute/path/to/project",
    "force": false
  }'

Run hybrid search:

curl -X POST http://127.0.0.1:8000/api/search \
  -H 'Content-Type: application/json' \
  -d '{
    "query": "Where is token validation implemented?",
    "repo_path": "/absolute/path/to/project",
    "top_k": 5,
    "mode": "hybrid",
    "rerank": true
  }'

Generate a walkthrough:

curl -X POST http://127.0.0.1:8000/api/synthesize \
  -H 'Content-Type: application/json' \
  -d '{
    "query": "Explain token validation failure paths",
    "repo_path": "/absolute/path/to/project",
    "top_k": 8,
    "provider": "extractive"
  }'

Important endpoints:

Bind to 127.0.0.1 unless remote access is intentionally required. Patch and file-opening endpoints operate on the local filesystem and should not be exposed to untrusted networks.

MCP integration

The MCP server lets VS Code, Cursor, Claude Code, and other compatible coding agents search the indexed codebase and retrieve exact source ranges. Install and index the project first:

git clone https://github.com/saitarrun/Semantic-code-intelligence.git
cd Semantic-code-intelligence
python -m venv .venv
source .venv/bin/activate
pip install -e .
code-intel index --dir /absolute/path/to/your/project

Use the absolute executable path printed by which code-intel in the examples below.

VS Code

Create .vscode/mcp.json in the project you want the agent to search:

{
  "servers": {
    "semanticCodeIntelligence": {
      "type": "stdio",
      "command": "/absolute/path/to/Semantic-code-intelligence/.venv/bin/code-intel",
      "args": ["mcp", "--dir", "${workspaceFolder}"],
      "cwd": "${workspaceFolder}"
    }
  }
}

Run MCP: List Servers from the Command Palette, start semanticCodeIntelligence, and approve its tools. If its old tool list is cached, run MCP: Reset Cached Tools.

Cursor

Create .cursor/mcp.json in the target project:

{
  "mcpServers": {
    "semantic-code-intelligence": {
      "command": "/absolute/path/to/Semantic-code-intelligence/.venv/bin/code-intel",
      "args": ["mcp", "--dir", "${workspaceFolder}"]
    }
  }
}

Claude Code

Register the local stdio server from the project you want to search:

claude mcp add --transport stdio --scope project semantic-code-intelligence -- \
  /absolute/path/to/Semantic-code-intelligence/.venv/bin/code-intel mcp --dir /absolute/path/to/your/project
claude mcp get semantic-code-intelligence

For another MCP-compatible agent, configure the same executable as a local stdio server with arguments mcp --dir /absolute/path/to/your/project. The server writes only JSON-RPC messages to stdout, as required by stdio clients.

Available MCP tools:

• code_intel_search: hybrid, dense, or sparse retrieval with exact lines and reliability metadata

• code_intel_symbol_graph: dependency and call-graph data for a repository or symbol

• code_intel_index: build or refresh an index from the coding agent

• code_intel_read_file: safely read up to 400 lines within the configured repository

The target project must be indexed before search requests. By default, its index is stored at <project>/.code_intel_index; pass --index-dir /path/to/index to the MCP command when using a separate index directory. Model downloads remain opt-in: set CODE_INTEL_ALLOW_MODEL_DOWNLOADS=1 if the embedding or reranker model is not already cached.

LSP and filesystem watcher

Start the stdio LSP bridge:

code-intel lsp --dir /absolute/path/to/project

Start the incremental watcher:

code-intel watch --dir /absolute/path/to/project

The watcher observes supported source files and refreshes index state after changes. Use Ctrl+C to stop either process.

Configuration

Environment variables:

Programmatic configuration:

from pathlib import Path

from semantic_code_intel.config import CodeIntelConfig
from semantic_code_intel.indexing.engine import HybridIndexer
from semantic_code_intel.retrieval.pipeline import HybridRetrievalPipeline

project = Path("/absolute/path/to/project")
config = CodeIntelConfig(project_root=project)
config.retrieval.dense_top_k = 75
config.retrieval.sparse_top_k = 75
config.retrieval.final_top_k = 8

HybridIndexer(config).index_codebase(project)
response = HybridRetrievalPipeline(config).query(
    "Where is request authentication enforced?",
    top_k=8,
)

for result in response.results:
    print(result.citation, result.chunk.symbol_name, result.score)

print(response.reliability, response.reliability_reasons)

Supported files

The default scanner includes:

• Python

• JavaScript and TypeScript

• Go

• Rust

• Java

• C and C++

• C#

• Ruby

• PHP

• Swift

• Kotlin and Scala

• Shell scripts

• SQL

• HTML and CSS

• JSON, YAML, TOML, and Markdown

Common generated directories, virtual environments, dependency folders, lock files, binaries, minified assets, .git, .code_intel_index, and oss_evaluation are excluded by default. See ParserConfig in semantic_code_intel/config.py to customize extensions and ignore patterns.

Architecture

flowchart LR
    A[Repository] --> B[Scanner and ignore rules]
    B --> C[Python AST or polyglot parser]
    C --> D[Symbol-aware chunks]
    D --> E[Local embedding model]
    E --> F[(FAISS)]
    D --> G[Code-aware tokenizer]
    G --> H[(BM25)]
    D --> I[(SQLite metadata)]

    Q[Query] --> X[Intent expansion]
    X --> F
    X --> H
    F --> R[Reciprocal Rank Fusion]
    H --> R
    R --> J[Cross-encoder reranker]
    J --> K[Exact symbol and path boosts]
    K --> L[Diversity and reliability]
    L --> M[CLI, API, Web, MCP, LSP]

Core modules:

Testing

Run the complete test suite:

uv run pytest -q

Or with an activated environment:

pytest -q

The suite covers parsers, FAISS, BM25, query expansion, exact-match boosting, fusion, citations, API endpoints, local synthesis behavior, MCP, LSP, patching, watching, and benchmark generation.

Benchmarking

Run a reproducible synthetic benchmark:

code-intel benchmark \
  --workspace ./benchmark_workspace \
  --loc 40000 \
  --queries 30

The runner writes benchmark_report.json containing:

• Dataset and index sizes

• Indexing throughput

• Dense, sparse, reranker, and end-to-end latency percentiles

• Hit rate and mean reciprocal rank

• Executed query records

• Python, platform, hardware, package, and model metadata

Benchmark results depend on hardware, model cache state, repository composition, and query set. Treat historical figures as measurements, not guarantees.

Troubleshooting

Model is not available locally

Run the failed operation once with downloads enabled:

CODE_INTEL_ALLOW_MODEL_DOWNLOADS=1 code-intel index /absolute/path/to/project --force
CODE_INTEL_ALLOW_MODEL_DOWNLOADS=1 code-intel query "warm up reranker" --dir /absolute/path/to/project

Index not found

The --dir and --index-dir values used for querying must match those used for indexing.

code-intel stats --dir /absolute/path/to/project

Walkthrough says Ollama is unavailable

Verify the local server and installed models:

ollama list
curl http://127.0.0.1:11434/api/tags

You can always use deterministic evidence mode:

code-intel ask "your question" --provider extractive

Search results are weak

• Use the exact class, function, method, endpoint, or configuration name when known.

• Prefer hybrid mode for normal use.

• Increase --top-k when the answer spans multiple files.

• Reindex with --force after changing parser or embedding configuration.

• Check the reliability label; low reliability means the retrieval signals do not strongly agree.

Server port is already in use

Choose another port:

code-intel serve --host 127.0.0.1 --port 8010

Project status

This project is under active development. Review generated patches before applying them, keep the API bound to localhost for normal use, and validate benchmark claims on your own target repositories.

License

No open-source license has been added yet. Public access to the repository does not by itself grant permission to copy, modify, or redistribute the code.