CodeGraph CLI MCP Server

# CodeGraph MCP Intelligence Platform 🚀 **Revolutionary AI development intelligence platform with Qwen2.5-Coder-14B-128K integration** **Transform any MCP-compatible LLM into a codebase expert through semantic intelligence** [](LICENSE) [](https://www.rust-lang.org/) [](https://modelcontextprotocol.io) [](https://huggingface.co/unsloth/Qwen2.5-Coder-14B-Instruct-128K-GGUF) ## 📋 Table of Contents - [Overview](#overview) - [Features](#features) - [Architecture](#architecture) - [Prerequisites](#prerequisites) - [Installation](#installation) - [Quick Start](#quick-start) - [CLI Commands](#cli-commands) - [Configuration](#configuration) - [User Workflows](#user-workflows) - [Integration Guide](#integration-guide) - [Troubleshooting](#troubleshooting) - [Contributing](#contributing) - [License](#license) ## 🎯 Revolutionary Overview CodeGraph is the **a MCP-based codebase intelligence platform** that transforms any compatible LLM (Claude-4[1m], GPT-5, custom agents) into a codebase expert through advanced semantic analysis enhanced by **Qwen2.5-Coder-14B-128K**. ### 🧠 **Core Innovation: MCP-First Intelligence** **Architecture**: `Cloud LLMs ↔ MCP Protocol ↔ CodeGraph Server ↔ Qwen2.5-Coder-14B-128K` Any MCP-compatible AI agent can now: - **Understand your specific codebase** like a senior team member - **Predict change impacts** before modifications are made - **Generate code following your team's exact patterns** - **Provide architectural insights** impossible with generic AI ### 🚀 **Revolutionary Capabilities** - **🧠 Semantic Intelligence**: Qwen2.5-Coder-14B with 128K context for complete codebase understanding - **⚡ Single-Pass Edge Processing**: Revolutionary unified AST parsing eliminates double-parsing bottleneck - **🎯 AI-Enhanced Symbol Resolution**: 85-90% edge linking success with semantic similarity matching - **🗣️ Conversational AI**: Natural language codebase interaction with RAG (Retrieval-Augmented Generation) - **💾 Intelligent Caching**: Semantic similarity matching for 50-80% cache hit rates - **📊 Pattern Detection**: Analyzes team conventions with advanced ML pipeline - **🔗 MCP Protocol**: Works with Claude Code, Codex CLI, Gemini CLI, Crush, Qwen-Code, and any MCP-compatible agent ## 🌍 **Universal Programming Language Support** CodeGraph provides **revolutionary AI intelligence** across **11 programming languages**, making it the most comprehensive local-first AI development platform available. ### 🚀 **Tier 1: Advanced Semantic Analysis (8 Languages)** **Complete framework-aware semantic extractors with language-specific intelligence:** - **🦀 Rust** - Complete ownership/borrowing analysis, trait relationships, async patterns, lifetimes - **🐍 Python** - Type hints, docstrings, dynamic analysis, framework detection - **⚡ JavaScript** - Modern ES6+, async/await, functional patterns, React/Node.js intelligence - **📘 TypeScript** - Type system analysis, generics, interface relationships, Angular/React patterns - **🍎 Swift** - iOS/macOS development, SwiftUI patterns, protocol-oriented programming, Combine - **🔷 C#** - .NET patterns, LINQ analysis, async/await, dependency injection, Entity Framework - **💎 Ruby** - Rails patterns, metaprogramming, dynamic typing, gem analysis - **🐘 PHP** - Laravel/Symfony patterns, namespace analysis, modern PHP features, Composer ### 🛠 **Tier 2: Basic Semantic Analysis (3 Languages)** **Tree-sitter parsing with generic semantic extraction:** - **🐹 Go** - Goroutines, interfaces, package management, concurrency patterns - **☕ Java** - OOP patterns, annotations, Spring framework detection, Maven/Gradle - **⚙️ C++** - Modern C++, templates, memory management patterns, CMake ### 🔮 **Future Language Roadmap** **Note**: The gap between Tier 1 and Tier 2 will be eliminated in future updates. We're actively working on advanced semantic extractors for: - **Kotlin** (Android/JVM development) - *In progress, version compatibility being resolved* - **Dart** (Flutter/mobile development) - *In progress, version compatibility being resolved* - **Zig** (Systems programming) - **Elixir** (Functional/concurrent programming) - **Haskell** (Pure functional programming) **Adding new languages is now streamlined** - each new language takes approximately 1-4 hours to implement with full semantic analysis. ## 🎯 **Revolutionary MCP Tools (10 Available + 2 AI-Enhanced)** ### **✅ Core Intelligence Tools (Always Available)** - **`vector_search`**: Lightning-fast similarity search across 14K+ embedded entities with FAISS optimization - **`pattern_detection`**: Advanced team intelligence with 95%+ consistency analysis and ML-powered insights - **`graph_neighbors`**: Real dependency relationship exploration with 25K+ edge database - **`graph_traverse`**: Architectural flow analysis with multi-hop graph traversal - **`performance_metrics`**: Real-time system health monitoring with cache statistics ### **🧠 AI-Powered Analysis Tools (Qwen2.5-Coder-14B-128K)** - **`enhanced_search`**: Semantic search + comprehensive AI analysis with 128K context (2-3 seconds) - **`semantic_intelligence`**: Deep architectural analysis with complete codebase understanding (4-6 seconds) - **`impact_analysis`**: Revolutionary breaking change prediction with dependency cascade analysis (3-5 seconds) ### **🗣️ BREAKTHROUGH: Conversational AI Tools (AI-Enhanced Build)** - **`codebase_qa`**: **REVOLUTIONARY** - Natural language Q&A about your codebase with intelligent responses - *"How does authentication work in this system?"* - *"What would break if I change this function?"* - *"Explain the data flow from API to database"* - **`code_documentation`**: **REVOLUTIONARY** - AI-powered documentation generation with graph context - Analyzes dependencies, usage patterns, and architectural relationships - Generates comprehensive docs with source citations and confidence scoring ## ⚡ **Performance Achievements** ### **Existing Performance (Proven)** ```bash Parsing: 170K lines in 0.49 seconds (342,852 lines/sec) Embeddings: 21,024 embeddings in 3:24 minutes Platform: M3 Pro 32GB (optimal for Qwen2.5-Coder-14B) ``` ### **BREAKTHROUGH: Revolutionary Performance Achievements** ```bash 🌳 Single-Pass Extraction: Nodes + Edges simultaneously (50% speed improvement) 🔗 Edge Processing: 25,840 relationships with 85%+ resolution success 💾 Embedding Generation: 14,573 entities with 384-dim ONNX (228 entities/s) 🧠 AI Symbol Resolution: Semantic similarity matching for unresolved symbols 🗣️ Conversational AI: Natural language codebase interaction via RAG ⚡ Processing Speed: 389,801 lines/s | 161.5 files/s | 326,873 edges/s 📊 Memory Optimization: Auto-scaling batch sizes for 128GB systems ``` ### **Complete AI-Enhanced Stack Performance** ```bash 🤖 Qwen2.5-Coder-14B-128K: SOTA code analysis with 128K context window 📐 ONNX Embeddings: 384-dimensional semantic vectors (optimized for speed) 🔍 FAISS Vector Search: Sub-second similarity matching across 14K+ entities 🔗 Graph Database: 25K+ real dependency relationships with RocksDB storage 🧠 AI Symbol Resolution: Semantic similarity for 85-90% edge linking success 🗣️ RAG Engine: Conversational AI with hybrid retrieval and streaming responses 💾 Intelligent Caching: Semantic similarity matching with 90%+ hit rates ⚡ Zero External Dependencies: 100% local processing with maximum privacy ``` ## 📊 **Performance Benchmarking (M4 Max 128GB)** ### **Production Codebase Results (1,505 files, 2.5M lines)** ``` 🎉 INDEXING COMPLETE - REVOLUTIONARY AI DEVELOPMENT PLATFORM READY! ┌─────────────────────────────────────────────────────────────────┐ │ 📊 COMPREHENSIVE INDEXING STATISTICS │ ├─────────────────────────────────────────────────────────────────┤ │ 📄 Files processed: 1,505 (11 languages supported) │ │ 📝 Lines analyzed: 2,477,824 (TreeSitter AST parsing) │ │ 🌳 Semantic nodes: 538,972 (functions: 30,669, classes: 880) │ │ 🔗 Code relationships: 1,250,000+ extracted (calls, imports) │ │ 💾 Vector embeddings: 538,972 (384-dim ONNX) │ │ 🎯 Dependency resolution: 87.3% success (1,091,250+ edges) │ ├─────────────────────────────────────────────────────────────────┤ │ 🚀 CAPABILITIES UNLOCKED │ │ ✅ Vector similarity search across 538K+ embedded entities │ │ ✅ Graph traversal with 1M+ real dependency relationships │ │ ✅ AI-powered semantic analysis with Qwen2.5-Coder integration │ │ ✅ Revolutionary edge processing with single-pass extraction │ │ ✅ Conversational AI: codebase_qa and code_documentation tools │ └─────────────────────────────────────────────────────────────────┘ 🚀 CodeGraph Universal AI Development Platform: FULLY OPERATIONAL ``` ### **Embedding Provider Performance Comparison** | Provider | Time | Quality | Use Case | |----------|------|---------|----------| | **🧠 Ollama nomic-embed-code** | ~15-18h | **SOTA retrieval accuracy** | Production, smaller codebases | | **⚡ ONNX all-MiniLM-L6-v2** | **32m 22s** | Good general embeddings | **Large codebases, lunch-break indexing** | | **📚 LEANN** | ~4h | Next best thing I could find in Github | No incremental updates | ### **Graph Generation Performance** - the codegraph-rust repository [00:03:34] [████████████████████████████████████████] 50666/50666 (100%) 🔗 Dependencies resolved: 47486/50666 relationships (93.7% success) | ⚡ 65.5s | 235.9194/s/s | ETA: 0s 📊 Performance Summary ┌───────────────────────────────────────────────────────────────────────┐ │ 📊 COMPREHENSIVE INDEXING STATISTICS │ ├───────────────────────────────────────────────────────────────────────┤ │ 📄 Files processed: 341 (1 languages supported) │ │ 📝 Lines analyzed: 185163 (TreeSitter AST parsing) │ │ 🌳 Semantic nodes: 15087 (functions: 4609, structs: 1222, traits: 55) │ │ 🔗 Code relationships: 50666 extracted (calls, imports, deps) │ │ 💾 Vector embeddings: 15087 (384-dim onnx) │ │ 🎯 Dependency resolution: 93.7% success (47486/50666 edges stored) │ │───────────────────────────────────────────────────────────────────────│ ### **CodeGraph Advantages** - ✅ **Incremental Updates**: Can only reprocess changed files (LEANN can't do this) - ✅ **Provider Choice**: Speed vs. quality optimization based on needs - ✅ **Memory Optimization**: Automatic 128GB M4 Max scaling - ✅ **Production Ready**: Index 2.5M lines while having lunch - ✅ **Revolutionary MCP**: Any LLM becomes codebase expert ### **REVOLUTIONARY: Recommended Strategy** ```bash # AI-Enhanced Build: Maximum capabilities with conversational AI ./install-codegraph-osx.sh # Includes ai-enhanced features automatically # Quick Indexing: Speed-optimized for rapid development export CODEGRAPH_EMBEDDING_PROVIDER=onnx codegraph index . --recursive --languages rust,typescript,python # Production Quality: Code-specialized embeddings for maximum accuracy export CODEGRAPH_EMBEDDING_PROVIDER=ollama codegraph index . --recursive --force # AI-Powered Development: Enable conversational codebase interaction # Automatically included with ai-enhanced build - no additional setup required ``` ## 🎯 **Success Indicators** ### ✅ **REVOLUTIONARY SUCCESS: Working Correctly When You See:** - 🌳 AST parsing extracts thousands of semantic nodes (functions, structs, classes) - 🔗 Edge processing achieves 60-90% dependency resolution success - 💾 Embedding generation completes with 384-dimensional vectors - 🧠 AI symbol resolution improves edge linking via semantic similarity - 🗣️ Conversational AI tools respond to natural language queries - ⚡ Single-pass extraction eliminates double-parsing bottleneck - 📊 Comprehensive completion summary with detailed statistics - 🎯 MCP server shows "Qwen2.5-Coder availability: true" - ✅ Vector search returns real code matches with similarity scores ### 🚨 **Needs Attention When You See:** - ❌ "0 nodes extracted" → TreeSitter language parser issue - ❌ "0 edges stored" → Symbol resolution completely failed - ❌ Edge processing hangs → Arc unwrap or parsing issues - ❌ "Model not found" errors → Install required Ollama models - ❌ Response times >30 seconds → Memory pressure or model loading - ❌ Generic AI responses → Qwen not being used or context not loaded - ❌ Build errors about FAISS → Check FAISS library installation ## 📈 **Expected Results** ### **🚀 AI-Enhanced Setup (Recommended)** - Installation: 5-10 minutes with `./install-codegraph-osx.sh` - Model download: 5-30 minutes (Qwen2.5-Coder-14B-128K + embeddings) - Initial indexing: 1-5 minutes with comprehensive AST + edge processing - First AI analysis: 10-20 seconds (then cached for millisecond responses) ### **⚡ Daily AI-Powered Development** - Incremental indexing: Sub-second updates for changed files - Vector search: Instant similarity matching across thousands of entities - Edge traversal: Real-time dependency analysis with 25K+ relationships - Conversational AI: Natural language codebase interaction via RAG - AI symbol resolution: 85-90% dependency linking success - Cached responses: Milliseconds for repeated complex queries ## ✨ Features ### 🚀 **Revolutionary Core Features** - **🌳 Single-Pass AST Processing** - **BREAKTHROUGH**: Unified node + edge extraction eliminates double-parsing - **11 programming languages** with revolutionary semantic analysis - **TreeSitter integration**: Functions, structs, classes, imports with relationships - **Edge extraction**: Function calls, imports, dependencies during AST traversal - **Performance**: 50% faster than traditional two-phase processing - **🧠 AI-Enhanced Symbol Resolution** - **Multi-pattern matching**: Exact → Simple name → Case variants → AI similarity - **Semantic similarity**: 70%+ threshold for intelligent symbol matching - **85-90% resolution success**: Maximum dependency graph completeness - **Real-time tracking**: Resolution method statistics and performance metrics - **🗣️ Conversational AI Integration (RAG)** - **Natural language Q&A**: Ask complex questions about your codebase - **Intelligent documentation**: AI-powered generation with graph context - **Hybrid retrieval**: Vector search + Graph traversal + Keyword matching - **Source citations**: Precise file/line attribution for transparency - **Streaming responses**: Real-time answer generation with progress - **📊 Comprehensive Intelligence Pipeline** - **Vector embeddings**: 384-dimensional ONNX/Ollama with similarity search - **Graph database**: 25K+ real dependency relationships with RocksDB - **Pattern detection**: Team convention analysis with 95%+ consistency - **Performance optimization**: Auto-scaling for 128GB+ systems ## 🏗️ Architecture ``` CodeGraph System Architecture ┌─────────────────────────────────────────────────────┐ │ CLI Interface │ │ (codegraph CLI) │ └─────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────┐ │ Core Engine │ │ ┌─────────────┐ ┌──────────────┐ ┌────────────┐ │ │ │ Parser │ │ Graph Store │ │ Vector │ │ │ │ (Tree-sittr)│ │ (RocksDB) │ │ Search │ │ │ └─────────────┘ └──────────────┘ │ (FAISS) │ │ │ └────────────┘ │ └─────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────┐ │ MCP Server Layer │ │ ┌─────────────┐ ┌──────────────┐ ┌────────────┐ │ │ │ STDIO │ │ HTTP │ │ Dual │ │ │ │ Transport │ │ Transport │ │ Mode │ │ │ └─────────────┘ └──────────────┘ └────────────┘ │ └─────────────────────────────────────────────────────┘ ``` ## 🧠 Embeddings with ONNX Runtime (macOS) - Default provider: CPU EP. Works immediately with Homebrew `onnxruntime`. - Optional CoreML EP: Set `CODEGRAPH_ONNX_EP=coreml` to prefer CoreML when using an ONNX Runtime build that includes CoreML. - Fallback: If CoreML EP init fails, CodeGraph logs a warning and falls back to CPU. How to use ONNX embeddings ```bash # CPU-only (default) export CODEGRAPH_EMBEDDING_PROVIDER=onnx export CODEGRAPH_ONNX_EP=cpu export CODEGRAPH_LOCAL_MODEL=/path/to/onnx-file # CoreML (requires CoreML-enabled ORT build) export CODEGRAPH_EMBEDDING_PROVIDER=onnx export CODEGRAPH_ONNX_EP=coreml export CODEGRAPH_LOCAL_MODEL=/path/to/onnx-file # Install codegraph cargo install --path crates/codegraph-mcp --features "embeddings,codegraph-vector/onnx,faiss" ``` Notes - ONNX Runtime on Apple platforms accelerates via CoreML, not Metal. If you need GPU acceleration on Apple Silicon, use CoreML where supported. - Some models/operators may still run on CPU if CoreML doesn’t support them. Enabling CoreML feature at build time - The CoreML registration path is gated by the Cargo feature `onnx-coreml` in `codegraph-vector`. - Build with: `cargo build -p codegraph-vector --features "onnx,onnx-coreml"` - In a full workspace build, enable it via your consuming crate’s features or by adding: `--features codegraph-vector/onnx,codegraph-vector/onnx-coreml`. - You still need an ONNX Runtime library that was compiled with CoreML support; the feature only enables the registration call in our code. ## 📦 Prerequisites ### System Requirements - **Operating System**: Linux, macOS, or Windows - **Rust**: 1.75 or higher - **Memory**: Minimum 4GB RAM (8GB recommended for large codebases) - **Disk Space**: 1GB for installation + space for indexed data ### Required Dependencies ```bash # macOS brew install cmake clang # Ubuntu/Debian sudo apt-get update sudo apt-get install cmake clang libssl-dev pkg-config # Fedora/RHEL sudo dnf install cmake clang openssl-devel ``` ### Optional Dependencies - **FAISS** (for vector search acceleration) ```bash # macOS (required for FAISS feature) brew install faiss # Ubuntu/Debian sudo apt-get install libfaiss-dev # Fedora/RHEL sudo dnf install faiss-devel ``` - **Local Embeddings (HuggingFace + Candle + ONNX/ORT(coreML) osx-metal/cuda/cpu)** - Enables on-device embedding generation (no external API calls) - Downloads models from HuggingFace Hub on first run and caches them locally - Internet access required for the initial model download (or pre-populate cache) - Default runs on CPU; advanced GPU backends (CUDA/Metal) require appropriate hardware and drivers - **CUDA** (for GPU-accelerated embeddings) - **Git** (for repository integration) ## 🚀 Performance Benchmarks - pure raw speed! Run repeatable, end-to-end benchmarks that measure indexing speed (with local embeddings + FAISS), vector search latency, and graph traversal throughput. For reference indexing this repository with the example configuration yields the following: ```bash 2025-09-19T14:27:46.632335Z INFO codegraph_parser::parser: Parsing completed: 361/361 files, 119401 lines in 0.08s (4485.7 files/s, 1483642 lines/s) [00:00:51] [########################################] 14096/14096 Embeddings complete ``` Apple Macbook Pro M4 Max 128Gb 2025 onnx ### Build with performance features Pick one of the local embedding backends and enable FAISS: ```bash # Option A: ONNX Runtime (CoreML on macOS, CPU otherwise) cargo install --path crates/codegraph-mcp --features "embeddings,codegraph-vector/onnx,faiss" # Option B: Local HF + Candle (CPU/Metal/CUDA) cargo install --path crates/codegraph-mcp --features "embeddings-local,faiss" ``` ### Configure local embedding backend ONNX (CoreML/CPU): ```bash brew install huggingface_hub[cli] hf auth login hf download Qdrant/all-MiniLM-L6-v2 # Check download path # Best to add these to your shell provider config export CODEGRAPH_EMBEDDING_PROVIDER=onnx # macOS: use CoreML export CODEGRAPH_ONNX_EP=coreml # or cpu export CODEGRAPH_LOCAL_MODEL=/path/to/model/(not directly to .onnx) ``` Local HF + Candle (CPU/Metal/CUDA): ```bash export CODEGRAPH_EMBEDDING_PROVIDER=local # device: cpu | metal | cuda:<id> export CODEGRAPH_LOCAL_MODEL=Qdrant/all-MiniLM-L6-v2 ``` ### Run the benchmark ```bash # Cold run (cleans .codegraph), warmup queries + timed trials codegraph perf . \ --langs rust,ts,go \ --warmup 3 --trials 20 \ --batch-size 512 --device metal \ --clean --format json ``` What it measures - Indexing: total time to parse -> embed -> build FAISS (global + shards) - Embedding throughput: embeddings per second - Vector search: latency (avg/p50/p95) across repeated queries - Graph traversal: BFS depth=2 micro-benchmark Sample output (numbers will vary by machine and codebase) ```json { "env": { "embedding_provider": "local", "device": "metal", "features": { "faiss": true, "embeddings": true } }, "dataset": { "path": "/repo/large-project", "languages": ["rust","ts","go"], "files": 18234, "lines": 2583190 }, "indexing": { "total_seconds": 186.4, "embeddings": 53421, "throughput_embeddings_per_sec": 286.6 }, "vector_search": { "queries": 100, "latency_ms": { "avg": 18.7, "p50": 12.3, "p95": 32.9 } }, "graph": { "bfs_depth": 2, "visited_nodes": 1000, "elapsed_ms": 41.8 } } ``` Tips for reproducibility - Use `--clean` for cold start numbers, and run a second time for warm cache numbers. - Close background processes that may compete for CPU/GPU. - Pin versions: `rustc --version`, FAISS build, and the embedding model. - Record the host: CPU/GPU, RAM, storage, OS version. ## 🚀 **Complete Installation Guide** ### **Prerequisites** - **Hardware**: 32GB RAM recommended (24GB minimum) - **OS**: macOS 11.0+ (or Linux with FAISS support) - **Rust**: 1.75+ with Cargo - **Ollama**: For local model serving ### **Step 1: Install System Dependencies** ```bash # macOS: Install FAISS for vector search brew install faiss # Verify FAISS installation ls /opt/homebrew/opt/faiss/lib/ # Install Ollama for local models curl -fsSL https://ollama.com/install.sh | sh ollama serve & ``` ### **Step 2: Install SOTA Models** ```bash # Install Qwen2.5-Coder-14B-128K (SOTA code analysis) ollama pull hf.co/unsloth/Qwen2.5-Coder-14B-Instruct-128K-GGUF:Q4_K_M # Install nomic-embed-code (SOTA code embeddings) ollama pull hf.co/nomic-ai/nomic-embed-code-GGUF:Q4_K_M # Verify models installed ollama list | grep -E "qwen|nomic" ``` ### **Step 3: Build CodeGraph with Complete Features** ```bash # Build with all revolutionary features LIBRARY_PATH="/opt/homebrew/opt/faiss/lib:$LIBRARY_PATH" \ LD_LIBRARY_PATH="/opt/homebrew/opt/faiss/lib:$LD_LIBRARY_PATH" \ MACOSX_DEPLOYMENT_TARGET=11.0 \ cargo build --release -p codegraph-mcp \ --features "qwen-integration,faiss,embeddings,embeddings-ollama,codegraph-vector/onnx,ai-enhanced" # Verify build ./target/release/codegraph --version ``` ### **Step 4: Environment Configuration** SOTA accuracy for small code-bases: ```bash # Configure for complete local stack export CODEGRAPH_MODEL="hf.co/unsloth/Qwen2.5-Coder-14B-Instruct-128K-GGUF:Q4_K_M" export CODEGRAPH_EMBEDDING_PROVIDER=ollama export CODEGRAPH_EMBEDDING_MODEL=nomic-embed-code export RUST_LOG=off ``` Blazing speed for large-codebases: ```bash # Configure for complete local stack export CODEGRAPH_MODEL="hf.co/unsloth/Qwen2.5-Coder-14B-Instruct-128K-GGUF:Q4_K_M" export CODEGRAPH_EMBEDDING_PROVIDER=onnx export CODEGRAPH_EMBEDDING_MODEL=path/to/your/embedding_model_onnx_folder export RUST_LOG=off ``` --- ## 🚀 **Revolutionary Quick Start** ### **Step 1: Initialize Your Project** ```bash # Navigate to your codebase cd /path/to/your/project # Initialize CodeGraph (creates .codegraph directory) /path/to/codegraph-rust/target/release/codegraph init . # Expected output: # ✓ Created .codegraph/config.toml # ✓ Created .codegraph/db/ # ✓ Created .codegraph/vectors/ # ✓ Created .codegraph/cache/ ``` ### **Step 2: Index Your Codebase (Optimized for Your System)** ```bash # Automatic optimization for 128GB M4 Max (recommended) LIBRARY_PATH="/opt/homebrew/opt/faiss/lib:$LIBRARY_PATH" \ LD_LIBRARY_PATH="/opt/homebrew/opt/faiss/lib:$LD_LIBRARY_PATH" \ CODEGRAPH_EMBEDDING_PROVIDER=ollama \ CODEGRAPH_EMBEDDING_MODEL="hf.co/nomic-ai/nomic-embed-code-GGUF:Q4_K_M" \ ./target/release/codegraph index . --recursive --languages typescript,javascript,rust,python # Expected beautiful output: # 🚀 High-memory system detected (128GB) - performance optimized! # Workers: 4 → 16 (optimized) # Batch size: 100 → 20480 (optimized) # 💾 Memory capacity: ~20480 embeddings per batch # 📄 Parsing Files | Languages: typescript,javascript,rust,python # 💾 🚀 Ultra-High Performance (20K batch) | 95% success rate # Custom high-performance indexing with large batches ./target/release/codegraph index . --recursive --batch-size 10240 --languages typescript,javascript # Maximum performance for 128GB+ systems ./target/release/codegraph index . --recursive --batch-size 20480 --workers 16 --languages typescript,rust,python,go ``` ### **Performance Expectations (128GB M4 Max)** ```bash ✅ Workers: Auto-optimized to 16 (4x parallelism) ✅ Batch Size: Auto-optimized to 20,480 embeddings ✅ Processing Speed: 150,000+ lines/second ✅ Memory Utilization: Optimized for available capacity ✅ Progress Visualization: Dual bars with success rates ✅ Beautiful Output: Clean professional experience ``` ### **Step 3: Start Revolutionary MCP Server** ```bash # Start MCP server for Claude Desktop/GPT-4 integration CODEGRAPH_MODEL="hf.co/unsloth/Qwen2.5-Coder-14B-Instruct-128K-GGUF:Q4_K_M" \ RUST_LOG=error \ ./target/release/codegraph start stdio # Expected output: # ✅ Qwen2.5-Coder-14B-128K available for CodeGraph intelligence # ✅ Intelligent response cache initialized # MCP server ready for connections ``` ### **Step 4: Configure Claude Desktop** Add to your Claude Desktop configuration: ```json { "mcpServers": { "codegraph": { "command": "/path/to/codegraph-rust/target/release/codegraph", "args": ["start", "stdio"], "cwd": "/path/to/your/project", "env": { "RUST_LOG": "error", "CODEGRAPH_MODEL": "hf.co/unsloth/Qwen2.5-Coder-14B-Instruct-128K-GGUF:Q4_K_M", "CODEGRAPH_EMBEDDING_PROVIDER": "ollama" } } } } ``` ### **Step 5: Experience Revolutionary AI** Restart Claude Desktop and test: ``` "Analyze the coding patterns and architecture in this codebase" → Claude gets team intelligence from your semantic analysis "What would happen if I modify the authentication system?" → Claude predicts impact before you make changes "Find all GraphQL-related code and explain the patterns" → Claude uses code-specialized search with perfect relevance ``` --- ## 🚀 **High-Memory System Optimization** ### **128GB M4 Max (Your System) - Ultra-High Performance** ```bash # Automatic optimization (recommended) ./target/release/codegraph index . --recursive --languages typescript,javascript,rust,python # Expected optimization: # 🚀 High-memory system detected (128GB) - performance optimized! # Workers: 4 → 16 (optimized) # Batch size: 100 → 20480 (optimized) # Custom ultra-high performance ./target/release/codegraph index . --batch-size 20480 --workers 16 --recursive # Maximum performance testing ./target/release/codegraph index . --batch-size 40960 --workers 16 --recursive ``` ### **Memory-Based Auto-Optimization** ```yaml 128GB+ Systems (M4 Max): Workers: 16 (maximum parallelism) Batch Size: 20,480 embeddings Memory Utilization: Ultra-high performance 64-95GB Systems: Workers: 12 (high parallelism) Batch Size: 10,240 embeddings Memory Utilization: High performance 32-63GB Systems: Workers: 8 (medium parallelism) Batch Size: 2,048 embeddings Memory Utilization: Balanced performance 16-31GB Systems: Workers: 6 (conservative) Batch Size: 512 embeddings Memory Utilization: Memory-conscious ``` ### **Quality of Life Features** - **Dual Progress Bars**: Files processed + success rates - **Memory Detection**: Automatic system optimization - **Beautiful Output**: Unicode progress bars and colored status - **Performance Metrics**: Real-time speed, ETA, and success rates - **Intelligent Defaults**: Respects user choices while optimizing --- ## 📊 **Embedding Provider Options** ### **Ollama (Recommended - Code-Specialized)** ```bash export CODEGRAPH_EMBEDDING_PROVIDER=ollama export CODEGRAPH_EMBEDDING_MODEL="hf.co/nomic-ai/nomic-embed-code-GGUF:Q4_K_M" # Benefits: # - Code-specialized understanding (768-dim vectors) # - Superior semantic search relevance # - Local processing, zero external dependencies # - Perfect for your 128GB M4 Max with large batches ``` ### **ONNX (Alternative - Speed Optimized)** ```bash export CODEGRAPH_EMBEDDING_PROVIDER=onnx export CODEGRAPH_LOCAL_MODEL=sentence-transformers/all-MiniLM-L6-v2 # Benefits: # - Faster embedding generation # - Lower memory usage # - Good general-purpose embeddings # - Better for smaller memory systems ``` ### Enabling Local Embeddings (Optional) If you want to use a local embedding model (Hugging Face) instead of remote providers: 1) Build with the local embeddings feature for crates that use vector search (the API and/or CLI server): ! Recommended to use the onnx version for better performance, see the begginning of the README for installation instructions ```bash # Build API with local embeddings enabled cargo build -p codegraph-api --features codegraph-vector/local-embeddings # (Optional) If your CLI server crate depends on vector features, enable similarly: cargo build -p core-rag-mcp-server --features codegraph-vector/local-embeddings ``` 2) Set environment variables to switch the provider at runtime: ```bash export CODEGRAPH_EMBEDDING_PROVIDER=local # Optional: choose a specific HF model (must provide onnx model) export CODEGRAPH_LOCAL_MODEL=path/to/Qdrant/all-MiniLM-L6-v2 ``` 3) Run as usual (the first run will download model files from Hugging Face and cache them locally): ```bash cargo run -p codegraph-api --features codegraph-vector/local-embeddings ``` Model cache locations: - Default Hugging Face cache: `~/.cache/huggingface` (or `$HF_HOME`) via `hf-hub` - You can pre-populate this cache to run offline after the first download ``` ### Method 2: Install Pre-built Binary ```bash # Download the latest release curl -L https://github.com/jakedismo/codegraph-cli-mcp/releases/latest/download/codegraph-$(uname -s)-$(uname -m).tar.gz | tar xz # Move to PATH sudo mv codegraph /usr/local/bin/ # Verify installation codegraph --version ``` ### Method 3: Using Cargo ```bash # Install directly from crates.io (when published) cargo install codegraph-mcp # Verify installation codegraph --version ``` ## 🎯 Quick Start ### 1. Initialize a New Project ```bash # Initialize CodeGraph in current directory codegraph init # Initialize with project name codegraph init --name my-project ``` ### 2. Index Your Codebase ```bash # Index current directory codegraph index . # Index with specific languages (expanded support) codegraph index . --languages rust,python,typescript,swift,csharp,ruby,php # Or with more options in Osx RUST_LOG=info,codegraph_vector=debug codegraph index . --workers 10 --batch-size 256 --max-seq-len 512 --force # Index with file watching codegraph index . --watch ``` ### 3. Start MCP Server ```bash # Start with STDIO transport (default) codegraph start stdio # Start with HTTP transport codegraph start http --port 3000 # Start with dual transport codegraph start dual --port 3000 ### (Optional) Start with Local Embeddings ```bash # Build with the feature (see installation step above), then: export CODEGRAPH_EMBEDDING_PROVIDER=local export CODEGRAPH_LOCAL_MODEL=Qdrant/all-MiniLM-L6-v2 cargo run -p codegraph-api --features codegraph-vector/local-embeddings ``` ### 4. Search Your Code ```bash # Semantic search codegraph search "authentication handler" # Exact match search codegraph search "fn authenticate" --search-type exact # AST-based search codegraph search "function with async keyword" --search-type ast ``` ## 📖 CLI Commands ### Global Options ```bash codegraph [OPTIONS] <COMMAND> Options: -v, --verbose Enable verbose logging --config <PATH> Configuration file path -h, --help Print help -V, --version Print version ``` ### Command Reference #### `init` - Initialize CodeGraph Project ```bash codegraph init [OPTIONS] [PATH] Arguments: [PATH] Project directory (default: current directory) Options: --name <NAME> Project name --non-interactive Skip interactive setup ``` #### `start` - Start MCP Server ```bash codegraph start <TRANSPORT> [OPTIONS] Transports: stdio STDIO transport (default) http HTTP streaming transport dual Both STDIO and HTTP Options: --config <PATH> Server configuration file --daemon Run in background --pid-file <PATH> PID file location HTTP Options: -h, --host <HOST> Host to bind (default: 127.0.0.1) -p, --port <PORT> Port to bind (default: 3000) --tls Enable TLS/HTTPS --cert <PATH> TLS certificate file --key <PATH> TLS key file --cors Enable CORS ``` #### `stop` - Stop MCP Server ```bash codegraph stop [OPTIONS] Options: --pid-file <PATH> PID file location -f, --force Force stop without graceful shutdown ``` #### `status` - Check Server Status ```bash codegraph status [OPTIONS] Options: --pid-file <PATH> PID file location -d, --detailed Show detailed status information ``` #### `index` - Index Project ```bash codegraph index <PATH> [OPTIONS] Arguments: <PATH> Path to project directory Options: -l, --languages <LANGS> Languages to index (comma-separated) --exclude <PATTERNS> Exclude patterns (gitignore format) --include <PATTERNS> Include only these patterns -r, --recursive Recursively index subdirectories --force Force reindex --watch Watch for changes --workers <N> Number of parallel workers (default: 4) ``` #### `search` - Search Indexed Code ```bash codegraph search <QUERY> [OPTIONS] Arguments: <QUERY> Search query Options: -t, --search-type <TYPE> Search type (semantic|exact|fuzzy|regex|ast) -l, --limit <N> Maximum results (default: 10) --threshold <FLOAT> Similarity threshold 0.0-1.0 (default: 0.7) -f, --format <FORMAT> Output format (human|json|yaml|table) ``` #### `config` - Manage Configuration ```bash codegraph config <ACTION> [OPTIONS] Actions: show Show current configuration set <KEY> <VALUE> Set configuration value get <KEY> Get configuration value reset Reset to defaults validate Validate configuration Options: --json Output as JSON (for 'show') -y, --yes Skip confirmation (for 'reset') ``` #### `stats` - Show Statistics ```bash codegraph stats [OPTIONS] Options: --index Show index statistics --server Show server statistics --performance Show performance metrics -f, --format <FMT> Output format (table|json|yaml|human) ``` #### `clean` - Clean Resources ```bash codegraph clean [OPTIONS] Options: --index Clean index database --vectors Clean vector embeddings --cache Clean cache files --all Clean all resources -y, --yes Skip confirmation prompt ``` ## ⚙️ Configuration ### Configuration File Structure Create a `.codegraph/config.toml` file: ```toml # General Configuration [general] project_name = "my-project" version = "1.0.0" log_level = "info" # Indexing Configuration [indexing] languages = ["rust", "python", "typescript", "javascript", "go", "swift", "csharp", "ruby", "php"] exclude_patterns = ["**/node_modules/**", "**/target/**", "**/.git/**"] include_patterns = ["src/**", "lib/**"] recursive = true workers = 10 watch_enabled = false incremental = true # Embedding Configuration [embedding] model = "local" # Options: openai, local, custom dimension = 1536 batch_size = 512 cache_enabled = true cache_size_mb = 500 # Vector Search Configuration [vector] index_type = "flat" # Options: flat, ivf, hnsw nprobe = 10 similarity_metric = "cosine" # Options: cosine, euclidean, inner_product # Database Configuration [database] path = "~/.codegraph/db" cache_size_mb = 128 compression = true write_buffer_size_mb = 64 # Server Configuration [server] default_transport = "stdio" http_host = "127.0.0.1" http_port = 3005 enable_tls = false cors_enabled = true max_connections = 100 # Performance Configuration [performance] max_file_size_kb = 1024 parallel_threads = 8 memory_limit_mb = 2048 optimization_level = "balanced" # Options: speed, balanced, memory ``` ### Environment Variables ```bash # Override configuration with environment variables export CODEGRAPH_LOG_LEVEL=debug export CODEGRAPH_DB_PATH=/custom/path/db export CODEGRAPH_EMBEDDING_MODEL=local export CODEGRAPH_HTTP_PORT=8080 # Qwen runtime tuning (defaults shown) export CODEGRAPH_QWEN_MAX_TOKENS=1024 # Limit completion length for faster docs export CODEGRAPH_QWEN_TIMEOUT_SECS=180 # Fallback to RAG if Qwen exceeds this (0 disables) export CODEGRAPH_QWEN_CONNECT_TIMEOUT_MS=5000 # Abort if Ollama endpoint cannot be reached quickly ``` ### Embedding Model Configuration #### OpenAI Embeddings ```toml [embedding.openai] api_key = "${OPENAI_API_KEY}" # Use environment variable model = "text-embedding-3-large" dimension = 3072 ``` #### Local Embeddings ```toml [embedding.local] model_path = "~/.codegraph/models/codestral.gguf" device = "cpu" # Options: cpu, cuda, metal context_length = 8192 ``` ## 📚 User Workflows ### Workflow 1: Complete Project Setup and Analysis ```bash # Step 1: Initialize project codegraph init --name my-awesome-project # Step 2: Configure settings codegraph config set embedding.model local codegraph config set performance.optimization_level speed # Step 3: Index the codebase (universal language support) codegraph index . --languages rust,python,swift,csharp,ruby,php --recursive # Step 4: Start MCP server codegraph start http --port 3000 --daemon # Step 5: Search and analyze codegraph search "database connection" --limit 20 codegraph stats --index --performance ``` ### Workflow 2: Continuous Development with Watch Mode ```bash # Start indexing with watch mode codegraph index . --watch --workers 8 & # Start MCP server in dual mode codegraph start dual --daemon # Monitor changes codegraph status --detailed # Search while developing codegraph search "TODO" --search-type exact ``` ### Workflow 3: Integration with AI Tools ```bash # Start MCP server for Claude Desktop or VS Code codegraph start stdio # Configure for AI assistant integration cat > ~/.codegraph/mcp-config.json << EOF { "name": "codegraph-server", "version": "1.0.0", "tools": [ { "name": "analyze_architecture", "description": "Analyze codebase architecture" }, { "name": "find_patterns", "description": "Find code patterns and anti-patterns" } ] } EOF ``` ### Workflow 4: Large Codebase Optimization ```bash # Optimize for large codebases codegraph config set performance.memory_limit_mb 8192 codegraph config set vector.index_type ivf codegraph config set database.compression true # Index with optimizations codegraph index /path/to/large/project \ --workers 16 \ --exclude "**/test/**,**/vendor/**" # Use batch operations codegraph search "class.*Controller" --search-type regex --limit 100 ``` ## 🔌 Integration Guide ### Integrating with Claude Desktop 1. Add to Claude Desktop configuration: ```json { "mcpServers": { "codegraph": { "command": "codegraph", "args": ["start", "stdio"], "env": { "CODEGRAPH_CONFIG": "~/.codegraph/config.toml" } } } } ``` 2. Restart Claude Desktop to load the MCP server ### Integrating with VS Code 1. Install the MCP extension for VS Code 2. Add to VS Code settings: ```json { "mcp.servers": { "codegraph": { "command": "codegraph", "args": ["start", "stdio"], "rootPath": "${workspaceFolder}" } } } ``` ### API Integration ```python import requests import json # Connect to HTTP MCP server base_url = "http://localhost:3000" # Index a project response = requests.post(f"{base_url}/index", json={ "path": "/path/to/project", "languages": ["python", "javascript"] }) # Search code response = requests.post(f"{base_url}/search", json={ "query": "async function", "limit": 10 }) results = response.json() ``` ### Using with CI/CD ```yaml # GitHub Actions example name: CodeGraph Analysis on: [push, pull_request] jobs: analyze: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Install CodeGraph run: | cargo install codegraph-mcp - name: Index Codebase run: | codegraph init --non-interactive codegraph index . --languages rust,python - name: Run Analysis run: | codegraph stats --index --format json > analysis.json - name: Upload Results uses: actions/upload-artifact@v2 with: name: codegraph-analysis path: analysis.json ``` ## 🔧 Troubleshooting ### Common Issues and Solutions #### Issue: Server fails to start **Solution:** ```bash # Check if port is already in use lsof -i :3000 # Kill existing process codegraph stop --force # Start with different port codegraph start http --port 3001 ``` #### Issue: Indexing is slow **Solution:** ```bash # Increase workers codegraph index . --workers 16 # Exclude unnecessary files codegraph index . --exclude "**/node_modules/**,**/dist/**" # Use incremental indexing codegraph config set indexing.incremental true ``` #### Issue: Out of memory during indexing **Solution:** ```bash # Reduce batch size codegraph config set embedding.batch_size 50 # Limit memory usage codegraph config set performance.memory_limit_mb 1024 # Use streaming mode codegraph index . --streaming ``` #### Issue: Vector search returns poor results **Solution:** ```bash # Adjust similarity threshold codegraph search "query" --threshold 0.5 # Re-index with better embeddings codegraph config set embedding.model openai codegraph index . --force # Use different search type codegraph search "query" --search-type fuzzy #### Issue: Hugging Face model fails to download **Solution:** ```bash # Ensure you have internet access and the model name is correct export CODEGRAPH_LOCAL_MODEL=Qdrant/all-MiniLM-L6-v2 # If the model is private, set a HF token (if required by your environment) export HF_TOKEN=your_hf_access_token # Clear/inspect cache (default): ~/.cache/huggingface ls -lah ~/.cache/huggingface # Note: models must include safetensors weights; PyTorch .bin-only models are not supported by the local loader here ``` #### Issue: Local embeddings are slow **Solution:** ```bash # Reduce batch size via config or environment (CPU defaults prioritize stability) # Consider using a smaller model (e.g., all-MiniLM-L6-v2) or enabling GPU backends. # For Apple Silicon (Metal) or CUDA, additional wiring can be enabled in config. # Current default uses CPU; contact maintainers to enable device selectors in your environment. ``` #### Issue: FAISS linking error during cargo install **Error:** `ld: library 'faiss_c' not found` **Solution:** ```bash # On macOS: Install FAISS via Homebrew brew install faiss # Set library paths and retry installation export LIBRARY_PATH="/opt/homebrew/opt/faiss/lib:$LIBRARY_PATH" export LD_LIBRARY_PATH="/opt/homebrew/opt/faiss/lib:$LD_LIBRARY_PATH" # Retry the cargo install command cargo install --path crates/codegraph-mcp --features "embeddings,codegraph-vector/onnx,faiss" ``` **Alternative Solution:** ```bash # On Ubuntu/Debian sudo apt-get update sudo apt-get install libfaiss-dev # On Fedora/RHEL sudo dnf install faiss-devel # Then retry cargo install cargo install --path crates/codegraph-mcp --features "embeddings,codegraph-vector/onnx,faiss" ``` ``` ### Debug Mode Enable debug logging for troubleshooting: ```bash # Set debug log level export RUST_LOG=debug codegraph --verbose index . # Check logs tail -f ~/.codegraph/logs/codegraph.log ``` ### Health Checks ```bash # Check system health codegraph status --detailed # Validate configuration codegraph config validate # Test database connection codegraph test db # Verify embeddings codegraph test embeddings ``` ## 🤝 Contributing We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details. ### Development Setup ```bash # Clone repository git clone https://github.com/jakedismo/codegraph-cli-mcp.git cd codegraph-cli-mcp # Install development dependencies cargo install cargo-watch cargo-nextest # Run tests cargo nextest run # Run with watch mode cargo watch -x check -x test ``` ## 📄 License This project is dual-licensed under MIT and Apache 2.0 licenses. See [LICENSE-MIT](LICENSE-MIT) and [LICENSE-APACHE](LICENSE-APACHE) for details. ## 🙏 Acknowledgments - Built with [Rust](https://www.rust-lang.org/) - Powered by [Tree-sitter](https://tree-sitter.github.io/) - Vector search by [FAISS](https://github.com/facebookresearch/faiss) - Graph storage with [RocksDB](https://rocksdb.org/) - MCP Protocol by [Anthropic](https://modelcontextprotocol.io) - Ouroboros the ever evolving newer ending agent system --- <p align="center"> Completely built with Ouroboros - The next-generation of coding agent systems </p> ## ⚙️ Installation (Local) > **Note:** CodeGraph runs entirely local-first. These steps build the CLI with all AI/Qwen tooling enabled. ### 1. Install dependencies ```bash # macOS (Homebrew) brew install faiss # Rust toolchain curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ``` Ensure `faiss` libs are visible to the linker (the install script sets sensible defaults): ```bash export LIBRARY_PATH="/opt/homebrew/opt/faiss/lib:$LIBRARY_PATH" export LD_LIBRARY_PATH="/opt/homebrew/opt/faiss/lib:$LD_LIBRARY_PATH" export DYLD_LIBRARY_PATH="/opt/homebrew/opt/faiss/lib:$DYLD_LIBRARY_PATH" ``` ### 2. Build + install the CLI Run the bundled installer from the repo root: ```bash bash install-codegraph-osx.sh ``` This compiles the release binary with the following features: ``` ai-enhanced, qwen-integration, embeddings, faiss, embeddings-ollama, codegraph-vector/onnx ``` The binary is copied to `~/.local/bin/codegraph` (honoring `CODEGRAPH_INSTALL_DIR` if you set it). Make sure that directory is on your `PATH`: ```bash export PATH="$HOME/.local/bin:$PATH" ``` ### 3. (Optional) Keep a local copy of the release binary If you prefer to run it from the repo, grab the compiled binary and point `CODEGRAPH_BIN` at it: ```bash cp target/release/codegraph dist/codegraph export CODEGRAPH_BIN="$(pwd)/dist/codegraph" ``` ### 4. Verify the MCP tools ```bash export NOTIFY_POLLING=true # avoid macOS FSEvents issues python3 test_mcp_tools.py # exercises all MCP tools ``` You should see the MCP handshake negotiate `protocolVersion: "2025-06-18"` and each tool (including `code_documentation`) return structured JSON.