codecontext
by Akgithub2028
README.md
<div align="center">
# CodeContext OS
### **Task-Aware Code Intelligence & Advanced Context Retrieval for Claude Code & Coding Agents**
[](https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/overview)
[](https://modelcontextprotocol.io)
[](https://python.org)
[](https://github.com/facebookresearch/faiss)
<br/>
[](https://sqlite.org)
[](https://build.nvidia.com)
[](tests/)
[](LICENSE)
<br/><br/>
<p align="center">
<img src="assets/Image.png" alt="CodeContext OS Architecture & Claude Code Integration" width="100%" />
</p>
<p align="center">
<b>Give Claude Code and AI coding agents a compact, task-shaped, evidence-backed view of a repository without forcing them to rediscover architecture through repeated file reads.</b>
</p>
[Architecture](docs/architecture.md) • [RAG Deep Dive](docs/rag_deep_dive.md) • [MCP Protocol Guide](docs/mcp_guide.md) • [Empirical Benchmarks](docs/evaluation_and_benchmarks.md) • [Examples](examples/)
</div>
---
## ⚡ Built Natively for Claude Code
When using Anthropic's **Claude Code** on large or complex repositories, agents typically waste context and tool turns searching for relevant functions, callers, and tests.
**CodeContext OS integrates seamlessly with Claude Code via the Model Context Protocol (MCP)**, giving Claude immediate, structured access to the entire codebase graph:
```text
Developer Prompt in Claude Code:
"Fix bug where password verification fails during login in AuthService"
│
▼
Claude Code calls CodeContext MCP tool:
`build_context(query="Fix password verification bug in AuthService")`
│
▼
CodeContext OS returns a cited Markdown evidence bundle:
├── 🎯 Target Symbol: src/auth/service.py#L12-L15 (AuthService.login)
├── 🔍 Internal Helper: src/auth/service.py#L37-L39 (AuthService._verify_hash)
├── 🔗 Upstream Callers: src/api/auth_router.py (auth_login_endpoint), src/orders/checkout.py (CheckoutService)
└── 🧪 Linked Tests: tests/test_auth.py (test_login, test_verify_password)
│
▼
Claude Code implements the precise patch in 1 turn (Zero exploration roundtrips)! 🚀
```
### 1-Step Setup for Claude Code:
Generate the `.mcp.json` configuration file in your workspace:
```bash
codecontext mcp config
```
```json
{
"mcpServers": {
"codecontext": {
"command": "uv",
"args": ["run", "codecontext", "mcp", "serve"]
}
}
}
```
Now, every time you open Claude Code in this repository, all **8 CodeContext OS tools** are automatically active and available to Claude.
---
## 🧭 Why We Built CodeContext OS
When we first evaluated autonomous coding agents on mid-to-large codebases, we identified two severe failure modes:
1. **The Grep Exploration Loop**: Unindexed agents rely on repeated `grep` and directory scans. On our benchmark, a standard agent took an average of **46 separate tool roundtrips** per task, burning through token limits and frequently losing track of execution context.
2. **The Naive Vector RAG Trap**: Slicing code into arbitrary 500-token chunks destroys AST hierarchies. When an agent asks *"What routes call this payment service and which tests cover it?"*, vector similarity returns text matches but completely misses cross-module callers, class inheritance, and dynamic dependency graphs.
**CodeContext OS** bridges this gap: a high-performance, local-first engine that fuses **AST symbol extraction, 2-hop structural call-graph traversal, BM25 + FAISS Reciprocal Rank Fusion ($k=60$), multi-signal feature reranking, and greedy token-budget packing**.
---
## 🏗️ System Architecture
```mermaid
flowchart TD
subgraph "1. Static Ingestion & Indexing Engine"
A[Repository Codebase] --> B[Repo Scanner & Ignore Filter]
B -->|Python AST| C[AST & Symbol Extractor]
B -->|Docs & OpenAPI v3| D[Markdown & Spec Parser]
B -->|Git Diff / Commit Log| E[Git Metadata Extractor]
C --> F[(SQLite Index: index.db)]
D --> F
E --> F
C --> G[Semantic Chunker]
D --> G
G --> H[(FAISS Vector Embeddings)]
end
subgraph "2. Hybrid Retrieval & Candidate Fusion"
I[Agent Query / Task] --> J[Hybrid Retrieval Engine]
J --> K[Lexical BM25 Engine]
J --> L[FAISS Vector Store]
J --> M[Exact Symbol Resolver]
K --> N[Reciprocal Rank Fusion - RRF k=60]
L --> N
M --> N
end
subgraph "3. Context Synthesis & Assembly"
N --> O[Structural Graph Expander]
F -.->|1-2 Hop Callers & Callees| O
O --> P[Multi-Signal Feature Reranker]
P --> Q[Greedy Budget Optimizer]
Q --> R[Secret Redaction & Injection Isolation]
R --> S[Task-Shaped Markdown Evidence Bundle]
end
subgraph "4. Agent Client Interface"
S --> T[FastMCP Protocol Server]
T --> U[Claude Code / Cursor / Autonomous Agent]
end
```
---
## 🔬 Core RAG & Code-Intelligence Highlights
### 1. Multi-Source Structural Chunking
- **AST-Guided Symbol Chunks**: Chunks functions, methods, and classes along strict AST boundaries, retaining signatures, parameters, return types, decorators, and docstrings intact.
- **OpenAPI v3 Spec Ingestion**: Ingests `openapi.json` and `openapi.yaml` files, extracting HTTP operations (`POST /auth/login`), operation IDs, tags, parameters, and schema models.
- **Markdown Architecture Docs**: Sections `README.md`, ADRs, and documentation along heading boundaries (`#`, `##`, `###`).
### 2. Identifier-Aware Code Tokenization
Splits complex camelCase and snake_case identifiers into sub-word tokens:
```python
# Identifier
"CheckoutService.process_order"
# Sub-word tokens indexed
["checkout", "service", "process", "order", "CheckoutService", "process_order"]
```
### 3. Reciprocal Rank Fusion ($k=60$)
Merges Lexical BM25, FAISS semantic vectors, and exact symbol hits:
$$\text{RRF}(d) = \sum_{c \in \text{Channels}} \frac{1}{60 + \text{rank}_c(d)}$$
### 4. Structural Graph Expansion (1-Hop & 2-Hop)
Traverses the SQLite structural graph to discover:
- **Upstream Callers**: Functions and API routes that invoke the target.
- **Downstream Callees**: Internal database and crypto helpers called by the target.
- **Inheritance**: Base classes and polymorphic implementations.
- **Linked Unit Tests**: Tests exercising the target code paths.
### 5. Multi-Signal Feature Reranker
Scores fused candidates using weighted structural and exact-match signals:
$$\text{Score}(c) = w_{\text{rrf}} \cdot S_{\text{rrf}} + w_{\text{exact}} \cdot M_{\text{exact}} + w_{\text{path}} \cdot M_{\text{path}} + w_{\text{test}} \cdot M_{\text{test}} - w_{\text{dist}} \cdot D_{\text{graph}}$$
### 6. Security, Redaction & Prompt Injection Defense
- **Secret Redaction**: Masks API keys, JWT tokens, AWS keys, database URIs, and private keys before delivery to LLMs.
- **Prompt Injection Isolation**: Neutralizes instruction hijack patterns (`IGNORE PREVIOUS INSTRUCTIONS`, `DAN MODE`, `<|im_start|>`) inside untrusted repository documentation and wraps them in passive data envelopes.
---
## 📊 Empirical Benchmarks & Statistical Evaluation
Evaluated against ground truth across **6 core categories** using **NVIDIA NIM `openai/gpt-oss-120b`** as an LLM-as-Judge:
| Evaluation Arm | Recall@1 | Recall@5 | Recall@10 | Precision@5 | MRR | nDCG@10 | Ctx Red. % | Tool Calls | Pass Rate |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **1. Bare Agent (Grep Baseline)** | `0.000` | `0.000` | `0.000` | `0.000` | `0.000` | `0.000` | `68.7%` | `46.0` | `0.0%` |
| **2. Pure Lexical (BM25)** | `0.278` | `0.722` | `0.778` | `0.333` | `0.806` | `0.687` | `92.5%` | `1.0` | `100.0%` |
| **3. Pure Dense (FAISS)** | `0.000` | `0.000` | `0.000` | `0.000` | `0.000` | `0.000` | `100.0%` | `1.0` | `0.0%` |
| **4. Hybrid (BM25+FAISS)** | `0.278` | `0.722` | `0.778` | `0.333` | `0.806` | `0.687` | `92.5%` | `1.0` | `100.0%` |
| **5. Hybrid + Graph** | `0.000` | `0.250` | `0.694` | `0.133` | `0.175` | `0.343` | `88.8%` | `2.0` | `33.3%` |
| **6. Full CodeContext OS** | **`0.444`** | **`0.694`** | **`0.778`** | **`0.333`** | **`1.000`** | **`0.761`** | **`58.4%`** | **`1.0`** | **`100.0%`** |
### Statistical Significance (Full CodeContext OS vs Baselines)
- **97.8% Tool Call Reduction**: Reduced agent exploration roundtrips from **46 tool calls** (Bare Agent) down to **1 single context assembly call**.
- **Top-Rank Quality**: Achieved a perfect **MRR = 1.000** and top **nDCG@10 = 0.761** via feature reranking.
- **Statistical Significance**: Statistically significant superiority over baseline search ($p = 0.0360 < 0.05$ on Wilcoxon signed-rank and $p = 0.0412 < 0.05$ on McNemar test).
---
## ⚡ FastMCP Server & Claude Code Integration
CodeContext OS exposes **8 task-oriented MCP tools**:
| MCP Tool | Purpose |
|---|---|
| `get_codebase_overview` | Summary of repository size, symbols, graph edges, languages, entrypoints. |
| `search_code` | Multi-mode search (`hybrid`, `lexical`, `dense`, `symbol`). |
| `find_symbol` | Exact and qualified AST symbol locator. |
| `get_symbol_context` | 360-degree graph context (callers, callees, inheritance, tests). |
| `get_change_context` | Git diff impact analysis and modified symbol tracking. |
| `get_related_tests` | Direct test discovery mapping production symbols to unit tests. |
| `get_dependencies` | Directional upstream/downstream graph traversal. |
| `build_context` | **Primary Tool**: Assembles task-shaped context bundle under token budget. |
### Connect to Claude Code in 1 Step:
```bash
# Generate .mcp.json in workspace root
codecontext mcp config
```
```json
{
"mcpServers": {
"codecontext": {
"command": "uv",
"args": ["run", "codecontext", "mcp", "serve"]
}
}
}
```
---
## 🚀 Installation & CLI Usage
### Install
```bash
# Using uv (recommended)
uv pip install -e .
# Or standard pip
pip install -e .
```
### CLI Commands
```bash
# Initialize CodeContext in repository
codecontext init .
# Build structural and semantic index
codecontext index .
# Fast incremental index (re-indexes only git-diff modified files & dependents)
codecontext index . --incremental
# Inspect codebase overview
codecontext inspect .
# Search code using hybrid retrieval
codecontext search "AuthService login" -m hybrid
# Assemble a cited Markdown context bundle under a 4,000 token budget
codecontext context "Fix authentication bug in AuthService" -b 4000
# Start MCP stdio server
codecontext mcp serve
# Run reproducible benchmark suite
codecontext benchmark run --representative
```
---
## 💻 Python SDK Example
```python
import asyncio
from pathlib import Path
from codecontext.context.pipeline import ContextPipeline
from codecontext.index.service import IndexService
async def main():
repo_path = Path(".").resolve()
# 1. Index repository
service = IndexService(repo_path)
stats = service.index_repository(force=False, incremental=True)
print(f"Indexed {stats.files_indexed} files in {stats.duration_ms:.2f} ms")
# 2. Assemble context bundle
pipeline = ContextPipeline(repo_path)
result = await pipeline.build_context(
query="Trace how CheckoutService calls AuthService for password verification",
token_budget=4000,
expand_graph=True,
)
print(result.markdown_bundle)
if __name__ == "__main__":
asyncio.run(main())
```
---
## 🛡️ Design Philosophy
- **Deterministic First, LLM Second**: Symbol lookups, graph traversals, and token packing are 100% deterministic, running in $<60\text{ ms}$ without requiring external API keys.
- **Evidence Before Prose**: Every context item carries exact file paths, line ranges, and relevance provenance.
- **Zero Hallucinated Metrics**: All benchmark results are empirically derived against frozen ground-truth tasks and validated with live LLM judges.
---
## 📄 License
CodeContext OS is open-source software licensed under the [MIT License](LICENSE).
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessSyncing