# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with this MCP Memory Service repository.
> **📝 Personal Customizations**: You can create `CLAUDE.local.md` (gitignored) for personal notes, custom workflows, or environment-specific instructions. This file contains shared project conventions.
> **Information Lookup**: Files first, memory second, user last. See [`.claude/directives/memory-first.md`](.claude/directives/memory-first.md) for strategy. Comprehensive project context stored in memory with tags `claude-code-reference`.
## 🔴 Critical Directives
**IMPORTANT**: Before working with this project, read:
- **`.claude/directives/memory-tagging.md`** - MANDATORY: Always tag memories with `mcp-memory-service` as first tag
- **`.claude/directives/README.md`** - Additional topic-specific directives
## Overview
MCP Memory Service is a Model Context Protocol server providing semantic memory and persistent storage for Claude Desktop and 13+ AI applications. It uses vector embeddings for semantic search, supports multiple storage backends (SQLite-vec, Cloudflare, Hybrid), and includes advanced features like memory consolidation, quality scoring, and OAuth 2.1 team collaboration.
**Current Version:** v10.0.0 - Major API redesign with 64% tool consolidation (34→12 tools, 100% backwards compatible) - see [CHANGELOG.md](CHANGELOG.md) for details
> **🎯 v10.0.0 Milestone**: This major release represents a complete API consolidation - 34 tools unified into 12 with enhanced capabilities. All deprecated tools continue working with warnings until v11.0. See `docs/MIGRATION.md` for migration guide.
## Essential Commands
### Development Server
```bash
# MCP server (for Claude Desktop integration)
python -m mcp_memory_service.server
# HTTP API server (dashboard + REST API)
python scripts/server/run_http_server.py
# Both servers simultaneously
./start_all_servers.sh
# Quick update after git pull (RECOMMENDED)
./scripts/update_and_restart.sh
```
### Testing
```bash
# Run all tests (968 tests total)
pytest
# Run specific test file
pytest tests/storage/test_sqlite_vec.py
# Run with markers
pytest -m unit # Fast unit tests only
pytest -m integration # Integration tests (require storage)
pytest -m performance # Performance benchmarks
# Run with coverage
pytest --cov=src/mcp_memory_service --cov-report=html
# Pre-PR validation (MANDATORY before submitting PR)
bash scripts/pr/pre_pr_check.sh
```
### Building & Installation
```bash
# Install in editable mode (development)
pip install -e .
# Install with optional dependencies
pip install -e ".[full]" # All features
pip install -e ".[sqlite]" # SQLite with ONNX only
pip install -e ".[ml]" # Full ML capabilities
# Build package
python -m build
```
### Health Checks
```bash
# Quick health check
curl http://127.0.0.1:8000/api/health
# Comprehensive validation
python scripts/validation/validate_configuration_complete.py
# Backend configuration diagnostics
python scripts/validation/diagnose_backend_config.py
```
**Full command reference:** [scripts/README.md](scripts/README.md)
## Code Architecture
### High-Level Structure
```
src/mcp_memory_service/
├── server/ # MCP server layer (modular, cache-optimized)
├── server_impl.py # Main MCP handlers (35 tools)
├── storage/ # Storage backends (Strategy Pattern)
├── web/ # FastAPI dashboard + REST API + OAuth
├── services/ # Business logic (MemoryService orchestrator)
├── quality/ # AI quality scoring (multi-tier)
├── consolidation/ # Dream-inspired memory maintenance
├── embeddings/ # ONNX embeddings (sentence-transformers)
├── ingestion/ # Document loaders (PDF, DOCX, TXT, JSON)
├── models/ # Data models and schemas
└── utils/ # Utilities (health checks, startup orchestrator)
```
### MCP Server Layer (`server/`)
**Evolution:** Extracted from monolithic 5000+ line `server.py` to modular architecture (v8.59.0)
**Key Components:**
- **`server_impl.py`** - Main MemoryServer class with 35 MCP tool handlers
- **`cache_manager.py`** - Global caching for 534,628x performance boost
- **`client_detection.py`** - Adapts behavior for Claude Desktop vs LM Studio
- **`handlers/`** - Modular request handlers (memory, quality, consolidation, graph)
- **`logging_config.py`** - Client-aware logging
- **`environment.py`** - Python path setup, version checks
**Pattern:** Global singleton caching prevents redundant storage initialization across MCP tool calls.
### Storage Backend Architecture (`storage/`)
**Strategy Pattern** with 3 implementations sharing `BaseStorage` interface:
| Backend | File | Size | Performance | Use Case |
|---------|------|------|-------------|----------|
| **SQLite-Vec** | `sqlite_vec.py` | 116KB | 5ms reads | Development, single-user |
| **Cloudflare** | `cloudflare.py` | 85KB | Network-dependent | Cloud-only, edge deployment |
| **Hybrid** | `hybrid.py` | 84KB | 5ms local + cloud sync | **Production (RECOMMENDED)** |
**Key Features:**
- All implement `BaseStorage` interface (`base.py`)
- SQLite-Vec uses sqlite-vec extension for KNN semantic search
- Cloudflare uses D1 (SQL) + Vectorize (vector index)
- Hybrid: Local SQLite-Vec for reads, background Cloudflare sync
- Graph storage in `graph.py` (v8.51.0) - 30x query performance
**Embeddings:** ONNX model (sentence-transformers/all-MiniLM-L6-v2) for lightweight vector generation
### Web Layer (`web/`)
**FastAPI-based REST API and dashboard:**
- **`app.py`** - Main FastAPI application
- **`api/`** - REST endpoints mirroring MCP tools
- **`oauth/`** - OAuth 2.1 Dynamic Client Registration (v7.0.0+)
- **`sse.py`** - Server-Sent Events for real-time updates
- **`static/`** - Single-page dashboard application
**Key Pattern:** HTTP API provides same functionality as MCP tools for team collaboration.
### Quality System (`quality/`)
**Multi-tier AI quality scoring** (v8.45.0+):
| Tier | Provider | Latency | Cost | Use Case |
|------|----------|---------|------|----------|
| 1 | Local ONNX | 80-150ms | $0 | **DEFAULT** - Fast, private |
| 2 | Groq/Llama 3 | 500-800ms | $0.0015 | Fallback if local fails |
| 3 | Gemini 1.5 Flash | 1-2s | $0.01 | High-accuracy scoring |
**Files:**
- `onnx_ranker.py` - Local ML-based quality scoring
- `ai_evaluator.py` - Cloud LLM scoring (Groq, Gemini)
- `async_scorer.py` - Async quality evaluation orchestrator
- `implicit_signals.py` - Access count, recency signals
**Usage:** Quality scores (0.0-1.0) used in quality-boosted search and retention policies.
### Consolidation System (`consolidation/`)
**Dream-inspired memory maintenance** (v8.23.0+):
**Components:**
- `decay.py` - Exponential decay scoring (importance × recency)
- `association_discovery.py` - Find semantic relationships
- `relationship_inference.py` - **NEW (v9.3.0+):** Intelligent relationship type classification
- `compression.py` - Semantic clustering and merging
- `forgetting.py` - Quality-based archival (High: 365d, Medium: 180d, Low: 30-90d)
- `scheduler.py` - Automatic consolidation scheduling (daily/weekly/monthly)
**Relationship Inference Engine (v9.3.0+):**
- Multi-factor analysis: memory type combinations, content semantics, temporal patterns, contradictions
- Automatic classification: causes, fixes, contradicts, supports, follows, related
- Confidence scoring (0.0-1.0) with default threshold of 0.6
- Integrated into association discovery - new associations automatically get inferred relationship types
- Retroactive updates: Use `scripts/maintenance/update_graph_relationship_types.py` for existing relationships
**Pattern:** Runs via HTTP API (90% token reduction vs MCP tools) with APScheduler.
### Document Ingestion (`ingestion/`)
**Pluggable loader architecture:**
- **`base.py`** - Abstract `DocumentLoader` interface
- **`registry.py`** - Automatic loader selection by file extension
- **Loaders:** `pdf_loader.py`, `text_loader.py`, `json_loader.py`, `csv_loader.py`
- **`semtools_loader.py`** - Optional LlamaParse integration (enhanced PDF/DOCX/PPTX)
- **`chunker.py`** - Intelligent text chunking (1000 chars, 200 overlap)
**Pattern:** Registry pattern allows easy addition of new document types.
## Test Architecture
### Structure (968 tests)
```
tests/
├── api/ # API layer tests (compact types, operations)
├── storage/ # Backend-specific tests (sqlite_vec, cloudflare, hybrid)
├── server/ # MCP server handler tests (35 handlers)
├── consolidation/ # Memory maintenance tests
├── quality/ # Quality scoring tests
├── web/ # HTTP API and OAuth tests
├── conftest.py # Shared fixtures
└── pytest.ini # Test configuration
```
### Key Fixtures (`conftest.py`)
- **`temp_db_path`** - Temporary database directory (auto-cleanup)
- **`unique_content`** - Generate unique test content to avoid duplicates
- **`test_store`** - Auto-tags memories with `__test__` for cleanup
- **`TEST_MEMORY_TAG = "__test__"`** - Reserved tag for automatic test cleanup
### Test Markers (defined in `pytest.ini`)
```python
@pytest.mark.unit # Fast unit tests
@pytest.mark.integration # Integration tests (require storage)
@pytest.mark.performance # Performance benchmarks
@pytest.mark.asyncio # Async tests (auto-detected)
```
### Running Tests by Category
```bash
pytest -m unit # Unit tests only
pytest -m integration # Integration tests
pytest -m performance # Performance benchmarks
pytest -k "test_store" # Tests matching name pattern
```
## Configuration
### Environment Variables
**Quick Reference** (full list in `.env.example`):
```bash
# Storage Backend
export MCP_MEMORY_STORAGE_BACKEND=hybrid # hybrid|cloudflare|sqlite_vec
# Cloudflare (required for hybrid/cloudflare)
export CLOUDFLARE_API_TOKEN="your-token"
export CLOUDFLARE_ACCOUNT_ID="your-account"
export CLOUDFLARE_D1_DATABASE_ID="your-db-id"
export CLOUDFLARE_VECTORIZE_INDEX="mcp-memory-index"
# HTTP Server
export MCP_HTTP_ENABLED=true
export MCP_HTTP_PORT=8000
export MCP_API_KEY="your-secure-key"
# OAuth (v9.0.6+)
export MCP_OAUTH_STORAGE_BACKEND=sqlite # memory|sqlite
export MCP_OAUTH_SQLITE_PATH=./data/oauth.db
# Quality System (v8.45.0+)
export MCP_QUALITY_SYSTEM_ENABLED=true
# Consolidation (v8.23.0+)
export MCP_CONSOLIDATION_ENABLED=true
# SQLite Concurrent Access (CRITICAL for HTTP + MCP servers)
export MCP_MEMORY_SQLITE_PRAGMAS=journal_mode=WAL,busy_timeout=15000,cache_size=20000
```
**Configuration Precedence:** Environment variables > .env file > Global Claude Config > defaults
**Important:** After updating `.env`, always restart servers. Use `./scripts/update_and_restart.sh` for automated workflow.
**CRITICAL:** `MCP_MEMORY_SQLITE_PRAGMAS` must include `journal_mode=WAL` for concurrent access. Omitting WAL disables concurrent reads/writes and causes "database is locked" errors when HTTP server and MCP server run simultaneously.
### Claude Desktop Integration
**Recommended configuration** (`~/.claude/config.json`):
```json
{
"mcpServers": {
"memory": {
"command": "python",
"args": ["-m", "mcp_memory_service.server"],
"env": {
"MCP_MEMORY_STORAGE_BACKEND": "hybrid"
}
}
}
}
```
**Alternative:** Use `uv run memory server` or direct script path (see v6.17.0+ migration notes in README).
## Development Guidelines
### Code Quality Standards
**Three-layer quality strategy:**
1. **Pre-commit** (<5s) - Groq/Gemini complexity + security (blocks: complexity >8, security issues)
2. **PR Quality Gate** (10-60s) - `bash scripts/pr/pre_pr_check.sh` (blocks: security, health <50)
3. **Periodic Review** (weekly) - pyscn analysis + trend tracking
**Health Score Thresholds:**
- `<50`: 🔴 Release blocker (cannot merge)
- `50-69`: 🟡 Action required (refactor within 2 weeks)
- `70+`: ✅ Continue development
**Utility Modules Pattern** (v8.61.0 - Phase 3 Refactoring):
- Strategy Pattern: `utils/health_check.py` (5 strategies)
- Orchestrator Pattern: `utils/startup_orchestrator.py` (3 orchestrators)
- Processor Pattern: `utils/directory_ingestion.py` (3 processors)
- Analyzer Pattern: `utils/quality_analytics.py` (3 analyzers)
**Target:** All complexity A-B grade (complexity ≤8)
### Development Workflow
**Read first:**
- [`.claude/directives/development-setup.md`](.claude/directives/development-setup.md) - Editable install
- [`.claude/directives/pr-workflow.md`](.claude/directives/pr-workflow.md) - Pre-PR checks (MANDATORY)
- [`.claude/directives/refactoring-checklist.md`](.claude/directives/refactoring-checklist.md) - Refactoring safety
- [`.claude/directives/version-management.md`](.claude/directives/version-management.md) - Release workflow
**Quick workflow:**
1. `pip install -e .` - Install in editable mode
2. Make changes
3. `pytest` - Run tests
4. `bash scripts/pr/pre_pr_check.sh` - Pre-PR validation (MANDATORY)
5. Create PR - Use github-release-manager agent for releases
**Memory Tagging:** Always tag memories with `mcp-memory-service` as first tag (see `.claude/directives/memory-tagging.md`)
### Common Development Tasks
**Add a new MCP tool:**
1. Add handler method to `src/mcp_memory_service/server_impl.py`
2. Register tool in `MemoryServer.__init__` tool list
3. Add tests in `tests/server/test_handlers.py`
4. Update MCP schema if needed
**Add a new storage backend:**
1. Implement `BaseStorage` interface from `src/mcp_memory_service/storage/base.py`
2. Add factory method in `src/mcp_memory_service/storage/factory.py`
3. Add tests in `tests/storage/test_<backend>.py`
4. Update configuration options
**Add a new document loader:**
1. Implement `DocumentLoader` interface from `src/mcp_memory_service/ingestion/base.py`
2. Register loader in `src/mcp_memory_service/ingestion/registry.py`
3. Add tests in `tests/ingestion/test_<loader>.py`
**Improve memory ontology and relationship types:**
1. **Memory types:** Run `scripts/maintenance/improve_memory_ontology.py` to re-classify memory types using high-confidence patterns
2. **Relationship types:** Run `scripts/maintenance/update_graph_relationship_types.py` to infer relationship types for existing associations
3. **Test first:** Both scripts support `--dry-run` to preview changes before applying
4. **Cleanup:** Use `scripts/maintenance/cleanup_memories.py` to remove test memories and orphaned data
## Troubleshooting
### Common Issues
| Issue | Quick Fix |
|-------|-----------|
| Wrong backend showing | `python scripts/validation/diagnose_backend_config.py` |
| Port mismatch (hooks timeout) | Verify same port in `~/.claude/hooks/config.json` and server (default: 8000) |
| Schema validation errors after PR merge | Run `/mcp` in Claude Code to reconnect with new schema |
| Database lock errors | Add `journal_mode=WAL` to `MCP_MEMORY_SQLITE_PRAGMAS` in `.env`, restart servers |
| Tests failing after git pull | Run `./scripts/update_and_restart.sh` (installs deps, restarts server) |
**Comprehensive troubleshooting:** [docs/troubleshooting/hooks-quick-reference.md](docs/troubleshooting/hooks-quick-reference.md)
**Configuration validation:**
```bash
python scripts/validation/validate_configuration_complete.py # Comprehensive
python scripts/validation/diagnose_backend_config.py # Backend-specific
```
## Agent Integrations
**Workflow automation:**
- **github-release-manager** - Complete release workflow (version bump, CHANGELOG, PR creation)
- **amp-bridge** - Fast refactoring with Amp CLI
- **code-quality-guard** - Quality analysis before commits
- **gemini-pr-automator** - Automated PR reviews and fixes
**Usage:** See [`.claude/directives/agents.md`](.claude/directives/agents.md) for complete workflows.
## Key Design Patterns
1. **Strategy Pattern** - Storage backends, health checks, quality analytics
2. **Orchestrator Pattern** - Startup orchestrator, consolidation scheduler
3. **Processor Pattern** - Document ingestion, file processing
4. **Registry Pattern** - Document loaders, storage factory
5. **Singleton Pattern** - Global caching (storage, service instances)
## Performance Characteristics
**Key Metrics** (from production deployments):
- **5ms reads** - SQLite-Vec local storage
- **534,628x faster** - Global caching optimization (v8.26.0)
- **90% token reduction** - Consolidation via HTTP API vs MCP tools
- **85%+ trigger accuracy** - Natural memory triggers (v7.1.3+)
- **80-150ms** - Local ONNX quality scoring
## Documentation
**Where to find information:**
- **CLAUDE.md** (this file) - Development guide for Claude Code
- **README.md** - User-facing documentation, installation, features
- **CHANGELOG.md** - Version history, breaking changes, migrations
- **scripts/README.md** - Complete script reference
- **docs/** - Guides, troubleshooting, architecture specs
- **Wiki** - Comprehensive documentation (https://github.com/doobidoo/mcp-memory-service/wiki)
- **`.claude/directives/`** - Topic-specific directives for Claude Code
**When to update each:**
- **CLAUDE.md** - Architecture changes, new patterns, development workflows
- **README.md** - New features, installation changes, user-facing updates
- **CHANGELOG.md** - Every version bump (use github-release-manager agent)
- **Wiki** - Detailed guides, troubleshooting, tutorials
## Additional Resources
- **Storage Backends:** [`.claude/directives/storage-backends.md`](.claude/directives/storage-backends.md)
- **Hooks Configuration:** [`.claude/directives/hooks-configuration.md`](.claude/directives/hooks-configuration.md)
- **Quality System:** [`.claude/directives/quality-system-details.md`](.claude/directives/quality-system-details.md)
- **Consolidation:** [`.claude/directives/consolidation-details.md`](.claude/directives/consolidation-details.md)
- **Code Quality:** [`.claude/directives/code-quality-workflow.md`](.claude/directives/code-quality-workflow.md)
---
**Quick Start Checklist for New Contributors:**
1. ✅ Read this file (CLAUDE.md)
2. ✅ Read `.claude/directives/memory-tagging.md` (MANDATORY)
3. ✅ Run `pip install -e .` (editable install)
4. ✅ Run `pytest` (verify tests pass)
5. ✅ Read relevant directive files for your work area
6. ✅ Make changes and run `bash scripts/pr/pre_pr_check.sh` before PR
