Obsidian Elite RAG MCP Server

# Obsidian Elite RAG MCP Server [](https://python.org) [](LICENSE) [](https://modelcontextprotocol.io) An elite Retrieval-Augmented Generation (RAG) system that transforms Obsidian vaults into AI-paired cognitive workflow engines with advanced Graphiti knowledge graph integration. ## 🌟 Features ### 🧠 Multi-Layer RAG Architecture - **L1: Semantic Context** (30% weight) - Vector similarity search with OpenAI embeddings - **L2: Knowledge Graph** (25% weight) - Graphiti-powered entity and relationship retrieval - **L3: Graph Traversal** (15% weight) - NetworkX-based link traversal - **L4: Temporal Context** (15% weight) - Time-based relevance and freshness - **L5: Domain Specialization** (15% weight) - Context-aware retrieval - **L6: Meta-Knowledge** (remaining weight) - Knowledge about knowledge ### 🔗 Advanced Knowledge Graph - **27+ Entity Types**: concepts, people, organizations, technologies, methodologies, frameworks, algorithms, etc. - **40+ Relationship Types**: implements, uses, depends_on, extends, based_on, similar_to, integrates_with, etc. - **Dual-Graph Architecture**: Neo4j (structured) + NetworkX (unstructured backup) - **Automatic Entity Extraction**: Pattern matching and NLP-based entity recognition - **Relationship Detection**: Confidence scoring and validation ### 🚀 MCP Server Integration - **Claude Code Compatible**: Full Model Context Protocol server implementation - **Tool-based API**: Ingest, query, search knowledge graph, get entity context - **Real-time Status**: System health monitoring and database connection checks - **Async Processing**: High-performance concurrent operations ## 📋 Requirements - **Python 3.9+** - **Docker & Docker Compose** - **OpenAI API key** - **Obsidian vault** (optional but recommended) - **Neo4j Database** (handled by setup scripts) - **Qdrant Vector Database** (handled by setup scripts) ## 🛠️ Installation ### Option 1: Install from PyPI (Recommended) ```bash pip install obsidian-elite-rag-mcp ``` ### Option 2: Install from Source ```bash git clone https://github.com/aegntic/aegntic-MCP.git cd aegntic-MCP/obsidian-elite-rag pip install -e . ``` ## 🚀 Quick Start ### 1. System Setup ```bash # Initialize the system obsidian-elite-rag-cli setup # Start both databases (Qdrant + Neo4j) obsidian-elite-rag-cli start-databases # Or start manually with Docker docker run -d --name qdrant -p 6333:6333 -v $(pwd)/data/qdrant:/qdrant/storage qdrant/qdrant:latest docker run -d --name neo4j -p 7474:7474 -p 7687:7687 -v $(pwd)/data/neo4j:/data \ --env NEO4J_AUTH=neo4j/password --env NEO4J_PLUGINS='["apoc","graph-data-science"]' \ neo4j:5.14 ``` ### 2. Ingest Your Obsidian Vault ```bash # Ingest all markdown files obsidian-elite-rag-cli ingest /path/to/your/obsidian/vault # Check system status obsidian-elite-rag-cli status /path/to/your/obsidian/vault ``` ### 3. Start MCP Server ```bash # Start the MCP server for Claude Code integration obsidian-elite-rag-cli server ``` ### 4. Configure Claude Code Add to your Claude Code configuration (`~/.config/claude-code/config.json`): ```json { "mcpServers": { "obsidian-elite-rag": { "command": "obsidian-elite-rag-cli", "args": ["server"], "env": { "OPENAI_API_KEY": "your-openai-api-key" } } } } ``` ## 📖 Usage Examples ### CLI Usage ```bash # Query the RAG system obsidian-elite-rag-cli query "How does the RAG system work?" /path/to/vault # Search knowledge graph for entities obsidian-elite-rag-cli graph /path/to/vault --entity-query "machine learning" # Technical queries obsidian-elite-rag-cli query "JWT authentication patterns" /path/to/vault --query-type technical # Research queries obsidian-elite-rag-cli query "latest developments in LLMs" /path/to/vault --query-type research ``` ### MCP Server Tools (Claude Code) When connected to Claude Code, you'll have access to these tools: 1. **`ingest_vault`** - Ingest markdown files from an Obsidian vault 2. **`query_rag`** - Query the elite RAG system with multi-layer retrieval 3. **`search_knowledge_graph`** - Search the Graphiti knowledge graph for entities 4. **`get_entity_context`** - Get rich context for a specific entity 5. **`get_related_entities`** - Get entities related through relationships 6. **`get_system_status`** - Get system status and database connections Example in Claude Code: ``` @obsidian-elite-rag please ingest my vault at /Users/me/Documents/Obsidian @obsidian-elite-rag query "what are the key concepts in machine learning?" with vault path /Users/me/Documents/Obsidian @obsidian-elite-rag search_knowledge_graph for "neural networks" in vault /Users/me/Documents/Obsidian ``` ## 🏗️ Architecture ### System Components ``` ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ Obsidian │ │ Claude Code │ │ MCP Protocol │ │ Vault │◄──►│ Integration │◄──►│ Server │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ Elite RAG System │ ├─────────────────┬─────────────────┬─────────────────────────────┤ │ Semantic │ Knowledge │ Temporal & Domain │ │ Search │ Graph │ Specialization │ │ (Qdrant) │ (Neo4j) │ │ └─────────────────┴─────────────────┴─────────────────────────────┘ ``` ### Knowledge Graph Entity Types - **Core**: concept, person, organization, event, location - **Technical**: technology, algorithm, framework, system, application - **Process**: methodology, workflow, process, pattern - **Implementation**: tool, library, database, api, protocol - **Documentation**: standard, specification, principle, theory, model - **Architecture**: design, implementation, project, research ### Knowledge Graph Relationship Types - **Structural**: part_of, implements, extends, based_on, depends_on - **Semantic**: similar_to, contrasts_with, related_to, examples_of - **Functional**: uses, enables, requires, supports, improves - **Cognitive**: defines, describes, explains, demonstrates, teaches - **Development**: builds_on, applies_to, references, cites, tests - **Operational**: manages, monitors, deploys, configures, maintains ## 📊 Performance Characteristics - **Retrieval Speed**: <100ms for context-rich queries - **Knowledge Coverage**: 95%+ recall on domain-specific queries - **Entity Recognition**: 90%+ accuracy for concepts, people, organizations - **Relationship Extraction**: 85%+ accuracy for semantic relationships - **Graph Traversal**: <50ms for entity relationship queries up to depth 4 - **Automation Coverage**: 80%+ routine knowledge tasks automated ## 🔧 Configuration ### Environment Variables ```bash # Required OPENAI_API_KEY=your-openai-api-key # Optional (auto-configured by setup scripts) NEO4J_URI=bolt://localhost:7687 NEO4J_USER=neo4j NEO4J_PASSWORD=password QDRANT_HOST=localhost QDRANT_PORT=6333 ``` ### Configuration File The system uses `config/automation-config.yaml` for detailed configuration: ```yaml knowledge_graph: enabled: true provider: graphiti graphiti: neo4j_uri: bolt://localhost:7687 neo4j_user: neo4j neo4j_password: "password" rag_system: layers: semantic: weight: 0.3 similarity_threshold: 0.7 knowledge_graph: weight: 0.25 max_depth: 4 # ... other layers ``` ## 📚 Resources and Skills Development ### Knowledge Resources The system includes comprehensive knowledge resources to support advanced RAG operations: - **[Knowledge Graph Schemas](resources/knowledge-graph-schemas.md)**: Complete entity and relationship definitions with 27+ entity types and 40+ relationship types for structured knowledge representation. - **[Retrieval Patterns](resources/retrieval-patterns.md)**: Advanced multi-layer retrieval strategies and optimization techniques for different query types and use cases. - **[Vault Organization Patterns](resources/vault-organization-patterns.md)**: Best practices for structuring Obsidian vaults to maximize knowledge discovery and AI collaboration. ### AI Interaction Prompts Specialized prompts for optimal AI-human collaboration: - **[Retrieval Optimization](prompts/retrieval-optimization.md)**: System tuning and optimization prompts for enhancing retrieval accuracy and performance. - **[Vault Analysis](prompts/vault-analysis.md)**: Comprehensive vault content analysis and knowledge gap identification prompts. - **[AI Collaboration](prompts/ai-collaboration.md)**: Workflow prompts for effective human-AI knowledge partnership and co-creation. ### Skills Development Framework Progressive skill development paths for advanced knowledge management: - **[Knowledge Graph Analysis](skills/knowledge-graph-analysis.md)**: Master graph structure analysis, entity relationship mapping, and knowledge gap identification. - **[Retrieval Systems Optimization](skills/retrieval-systems-optimization.md)**: Advanced retrieval algorithm optimization, performance tuning, and user experience enhancement. - **[Knowledge Management Strategy](skills/knowledge-management-strategy.md)**: Strategic knowledge architecture design, capture strategies, and organizational learning frameworks. - **[Skills Overview](skills/skills.md)**: Complete framework guide with learning paths, competency development, and measurement approaches. ## 📁 Directory Structure ``` obsidian-elite-rag/ ├── src/obsidian_elite_rag/ # Core Python package │ ├── core/ # Core RAG components │ │ ├── rag_engine.py # Multi-layer RAG system │ │ └── graphiti_adapter.py # Graphiti integration │ ├── server.py # MCP server implementation │ ├── cli.py # Command-line interface │ └── scripts/ # Utility scripts ├── config/ # Configuration files ├── resources/ # Knowledge resources and schemas │ ├── knowledge-graph-schemas.md # 27+ entity types, 40+ relationship types │ ├── retrieval-patterns.md # Multi-layer retrieval strategies │ └── vault-organization-patterns.md # Optimal Obsidian vault structures ├── prompts/ # AI interaction prompts │ ├── retrieval-optimization.md # System optimization and tuning prompts │ ├── vault-analysis.md # Vault content analysis prompts │ └── ai-collaboration.md # AI-human collaboration workflows ├── skills/ # Skills development framework │ ├── knowledge-graph-analysis.md # Advanced graph analysis skills │ ├── retrieval-systems-optimization.md # Retrieval optimization skills │ ├── knowledge-management-strategy.md # KM strategy skills │ └── skills.md # Skills overview and framework ├── templates/ # Note templates ├── docs/ # Documentation ├── Dockerfile # Container configuration ├── docker-compose.yml # Multi-service deployment └── README.md # Main documentation ``` ### Vault Structure The system works best with this Obsidian vault structure: ``` 00-Core/ # Foundational knowledge 01-Projects/ # Active work 02-Research/ # Learning areas 03-Workflows/ # Reusable processes 04-AI-Paired/ # AI interactions 05-Resources/ # External references 06-Meta/ # System knowledge 07-Archive/ # Historical data 08-Templates/ # Note structures 09-Links/ # External connections ``` ## 🤝 Contributing We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details. ### Development Setup ```bash # Clone the repository git clone https://github.com/aegntic/aegntic-MCP.git cd aegntic-MCP/obsidian-elite-rag # Install in development mode pip install -e ".[dev]" # Run tests pytest # Run with coverage pytest --cov=obsidian_elite_rag # Code formatting black src/ mypy src/ ``` ## 📄 License This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. ## 🙏 Attribution **Created by:** Mattae Cooper **Email:** research@aegntic.ai **Organization:** Aegntic AI (https://aegntic.ai) This project represents advanced research in AI-powered knowledge management and retrieval-augmented generation systems. The integration of Graphiti knowledge graphs with multi-layered RAG architecture represents a significant advancement in how AI systems can interact with and reason over personal knowledge bases. ## 📞 Support - **Documentation**: [Project Wiki](https://github.com/aegntic/aegntic-MCP/wiki) - **Issues**: [GitHub Issues](https://github.com/aegntic/aegntic-MCP/issues) - **Discussions**: [GitHub Discussions](https://github.com/aegntic/aegntic-MCP/discussions) - **Email**: research@aegntic.ai ## 🔗 Related Projects - [Graphiti](https://github.com/getgraphiti/graphiti) - Knowledge graph construction for LLMs - [Qdrant](https://github.com/qdrant/qdrant) - Vector similarity search engine - [Neo4j](https://github.com/neo4j/neo4j) - Graph database - [LangChain](https://github.com/langchain-ai/langchain) - LLM application framework - [Model Context Protocol](https://github.com/modelcontextprotocol) - Standard for AI tool integration --- **Made with ❤️ by Aegntic AI** *Advancing the future of AI-powered knowledge management*