Skip to main content
Glama
mso-docs

Docs Navigator MCP - SUSE Edition

README.md
# Docs Navigator MCP - SUSE Edition

![Docs Navigator MCP - SUSE Edition](./assets/Docs%20Navigator%20MCP.png)

An **AI-powered documentation navigator** built as a Model Context Protocol (MCP) server that enables intelligent search, summarization, and exploration of SUSE, Rancher, K3s, RKE2, Longhorn, Harvester, NeuVector, and Kubewarden documentation using **open-source AI models**.

✨ **New:** Production-ready with SQLite caching, advanced analytics, concurrent indexing support, and organized codebase!

🎨 **[View Live Demo](https://mso-docs.github.io/Docs-Navigator-MCP-SUSE-Edition/)** - Check out the UI demo on GitHub Pages!

## πŸ“š Documentation

- [Architecture Overview](docs/ARCHITECTURE.md) - System design and components
- [Installation Guide](docs/INSTALL.md) - Detailed setup instructions
- [Documentation Sources](docs/SOURCES.md) - All available sources and usage guide
- [Indexing Guide](docs/INDEXING_GUIDE.md) - Understanding caching vs indexing ⭐
- [Quick Reference](docs/QUICKREF.md) - Command reference
- [Usage Examples](docs/EXAMPLES.md) - Common usage patterns
- [Web GUI Guide](docs/WEB_GUI.md) - Web interface documentation
- [MCP Client Setup](docs/MCP_CLIENT_CONFIG.md) - Configure MCP clients
- [SQLite Migration Guide](docs/SQLITE_MIGRATION.md) - SQLite caching details
- [Change Detection](docs/CHANGE_DETECTION.md) - Auto-update system for monitoring doc changes ⭐
- [GitHub Pages Deployment](docs/GITHUB_PAGES.md) - Deploy UI showcase to GitHub Pages
- [Contributing](docs/CONTRIBUTING.md) - Contribution guidelines

## 🌟 Features

### Core Capabilities
- 🌐 **Web GUI** - Beautiful localhost web interface for easy access
- πŸ” **Semantic Documentation Search** - Find relevant docs using natural language queries
- πŸ€– **Local Open-Source AI** - Powered by Ollama (Llama, Mistral, etc.) - no API keys required
- πŸ“š **Multi-Source Support** - Navigate SUSE, Rancher, K3s, RKE2, Longhorn, Harvester, NeuVector, Kubewarden documentation
- πŸ’¬ **Conversational Interface** - Ask questions and get answers with source citations
- πŸ“ **Smart Summarization** - Generate concise or detailed summaries of documentation
- πŸ”Œ **MCP Protocol** - Integrates with Claude Desktop and other MCP-compatible clients
- ⚑ **Vector Search** - Fast semantic retrieval using embeddings
- 🎯 **Flexible AI Providers** - Support for Ollama (local), OpenAI, or Anthropic

### Production Features (NEW!)

- πŸ’Ύ **SQLite Caching** - Efficient page caching with SQLite for scaling to 1000s of documents
- πŸ“Š **Analytics Reports** - Comprehensive cache analytics and health monitoring
- πŸ” **Advanced Queries** - Filter cached documents by source, status, date range
- πŸ”’ **Concurrent Indexing** - Safe multi-process indexing with automatic locking
- πŸ“ˆ **Cache Management** - Validate, clear, rebuild, and optimize caches
- 🎯 **Smart Indexing** - Conditional GET requests, ETag/Last-Modified support, content hash detection
- πŸ”„ **Change Detection** - Monitor docs for updates and auto-trigger re-indexing
- πŸ“¦ **Organized Codebase** - Clean directory structure with CLI/services/tests/utils separation

## πŸ—οΈ Architecture

```mermaid
graph TB
    subgraph "Client Layer"
        WEB[🌐 Web Browser<br/>localhost:3000]
        CLAUDE[πŸ€– Claude Desktop<br/>MCP Client]
        CLI[⌨️ CLI Tools<br/>npm commands]
    end

    subgraph "Application Layer"
        WEBSERVER[πŸ–₯️ Web Server<br/>Express.js<br/>Port 3000]
        MCP[πŸ“‘ MCP Server<br/>index.js<br/>stdio protocol]
        CLITOOLS[πŸ› οΈ CLI Tools<br/>indexer, analytics,<br/>cache manager]
    end

    subgraph "Service Layer"
        DOCSERVICE[πŸ“š Documentation Service<br/>- Fetch & Parse HTML<br/>- Content Extraction<br/>- Sitemap Processing]
        AISERVICE[🧠 AI Service<br/>- LLM Integration<br/>- Prompt Management<br/>- Response Formatting]
        VECTORSERVICE[πŸ” Vector Service<br/>- Embedding Generation<br/>- Semantic Search<br/>- Similarity Ranking]
        CACHESERVICE[πŸ’Ύ Cache Service<br/>- SQLite Operations<br/>- Page Management<br/>- Lock Handling]
    end

    subgraph "Storage Layer"
        SQLITE[(πŸ—„οΈ SQLite Database<br/>page-cache.db<br/>- Pages Table<br/>- Locks Table)]
        VECTORS[(πŸ“Š Vector Index<br/>vectra/index.json<br/>- Embeddings<br/>- Metadata)]
        HTMLCACHE[πŸ“ HTML Cache<br/>data/html/<br/>cached pages]
    end

    subgraph "External Services"
        OLLAMA[πŸ¦™ Ollama<br/>Local LLMs<br/>localhost:11434]
        OPENAI[🌐 OpenAI API<br/>GPT Models<br/>Embeddings]
        ANTHROPIC[πŸ”· Anthropic API<br/>Claude Models]
        DOCS[πŸ“– Documentation Sites<br/>- docs.k3s.io<br/>- ranchermanager.docs<br/>- documentation.suse.com<br/>- etc.]
    end

    %% Client connections
    WEB -->|HTTP/REST API| WEBSERVER
    CLAUDE -->|stdio/MCP| MCP
    CLI -->|Node.js| CLITOOLS

    %% Application layer connections
    WEBSERVER -->|Use Services| DOCSERVICE
    WEBSERVER -->|Use Services| AISERVICE
    WEBSERVER -->|Use Services| VECTORSERVICE
    MCP -->|Use Services| DOCSERVICE
    MCP -->|Use Services| AISERVICE
    MCP -->|Use Services| VECTORSERVICE
    CLITOOLS -->|Use Services| DOCSERVICE
    CLITOOLS -->|Use Services| CACHESERVICE

    %% Service layer connections
    DOCSERVICE -->|Read/Write| CACHESERVICE
    DOCSERVICE -->|Fetch| DOCS
    DOCSERVICE -->|Store HTML| HTMLCACHE
    AISERVICE -->|Query LLM| OLLAMA
    AISERVICE -->|Query LLM| OPENAI
    AISERVICE -->|Query LLM| ANTHROPIC
    VECTORSERVICE -->|Generate Embeddings| OLLAMA
    VECTORSERVICE -->|Generate Embeddings| OPENAI
    VECTORSERVICE -->|Read/Write| VECTORS
    CACHESERVICE -->|SQL Operations| SQLITE

    %% Styling
    classDef clientStyle fill:#e1f5ff,stroke:#01579b,stroke-width:2px
    classDef appStyle fill:#f3e5f5,stroke:#4a148c,stroke-width:2px
    classDef serviceStyle fill:#e8f5e9,stroke:#1b5e20,stroke-width:2px
    classDef storageStyle fill:#fff3e0,stroke:#e65100,stroke-width:2px
    classDef externalStyle fill:#fce4ec,stroke:#880e4f,stroke-width:2px

    class WEB,CLAUDE,CLI clientStyle
    class WEBSERVER,MCP,CLITOOLS appStyle
    class DOCSERVICE,AISERVICE,VECTORSERVICE,CACHESERVICE serviceStyle
    class SQLITE,VECTORS,HTMLCACHE storageStyle
    class OLLAMA,OPENAI,ANTHROPIC,DOCS externalStyle
```

### Key Components

- **Web GUI**: Modern React-like interface for interactive documentation search
- **MCP Server**: Model Context Protocol implementation for Claude Desktop integration
- **CLI Tools**: Command-line utilities for indexing, analytics, and cache management
- **Documentation Service**: Handles fetching, parsing, and caching of documentation pages
- **AI Service**: Manages LLM interactions with support for multiple providers (Ollama, OpenAI, Anthropic)
- **Vector Service**: Generates embeddings and performs semantic search using Vectra
- **Cache Service**: SQLite-based caching system with locking for concurrent operations

## πŸ†• Recent Updates (December 2025)

### Phase 3: Production-Ready SQLite Migration

**What's New:**

1. **SQLite Caching System**
   - Migrated from JSON to SQLite for better scalability
   - Support for 1000s of documents without performance degradation
   - Automatic schema creation with indexes on source, status, timestamps
   - Backward compatible - can revert to JSON with `USE_JSON_CACHE=true`

2. **Advanced Analytics & Queries**
   - `npm run analytics` - Comprehensive cache health reports
   - `npm run query-cache` - Filter by source, status, date range
   - Cache efficiency metrics, stale page detection, automated recommendations
   - By-source breakdowns showing indexed/total ratios

3. **Concurrent Indexing Safety**
   - Table-based locking prevents race conditions
   - 30-minute lock timeout with automatic expiration
   - `npm run clear-locks` for manual lock cleanup
   - Safe to run multiple indexing processes simultaneously

4. **Organized Codebase**
   - Clean directory structure: `cli/`, `services/`, `tests/`, `utils/`
   - 15 files reorganized from root into logical folders
   - Better maintainability and developer experience
   - Comprehensive documentation in `src/README.md`

5. **Performance Optimizations**
   - Smart caching with ETag/Last-Modified support
   - Content hash detection skips unchanged documents
   - Sitemap optimization pre-filters URLs before fetching
   - Conditional GET requests reduce unnecessary downloads

**Migration:**
- Existing users: Run `npm run migrate-sqlite` to convert JSON cache
- New users: SQLite is default, no action needed
- See [docs/SQLITE_MIGRATION.md](docs/SQLITE_MIGRATION.md) for details

## πŸš€ Quick Start

### Prerequisites

- **Node.js 18+** - [Download here](https://nodejs.org/)
- **Ollama** (for local AI) - [Download here](https://ollama.ai)
- **OpenAI API Key** (optional) - For production embeddings

### Step 1: Install Dependencies

```bash
# Clone the repository
git clone https://github.com/mso-docs/Docs-Navigator-MCP-SUSE-Edition.git
cd Docs-Navigator-MCP-SUSE-Edition

# Install Node.js dependencies
npm install
```

### Step 2: Install AI Models

**Option A: Local AI with Ollama (Free, Recommended for Q&A)**

```bash
# Install Ollama from https://ollama.ai

# Pull required models
ollama pull llama3.2:latest       # For answering questions
ollama pull nomic-embed-text      # For embeddings (alternative)

# Verify Ollama is running
ollama list
```

**Option B: OpenAI (Recommended for Embeddings)**

```bash
# Get API key from https://platform.openai.com/api-keys
# Add to .env file: OPENAI_API_KEY=your_key_here
```

**Hybrid Approach (Best Performance):**
- Use OpenAI for embeddings (fast, reliable)
- Use Ollama for Q&A (free, private)

### Step 3: Configure Environment

```bash
# Copy example environment file
cp .env.example .env

# Edit .env with your settings
nano .env  # or use your favorite editor
```

**Essential Configuration:**

```env
# For Ollama-only setup
AI_PROVIDER=ollama
EMBEDDING_PROVIDER=ollama
OLLAMA_MODEL=llama3.2:latest
EMBEDDING_MODEL=nomic-embed-text

# For OpenAI embeddings + Ollama Q&A (Recommended)
AI_PROVIDER=ollama
EMBEDDING_PROVIDER=openai
OPENAI_API_KEY=sk-your-key-here
OLLAMA_MODEL=llama3.2:latest

# Cache settings (SQLite is default)
PAGE_CACHE_PATH=./data/page-cache.db
EMBEDDING_CACHE_PATH=./data/embedding-cache.json
```

### Step 4: Index Documentation

```bash
# Index all documentation sources (SUSE, Rancher, K3s)
npm run index all

# Or index individual sources
npm run index k3s
npm run index rancher
npm run index suse
```

**First-time indexing takes 5-15 minutes** depending on your internet speed and AI provider. Subsequent runs are much faster due to caching.

### Step 5: Start Using!

**Web Interface (Easiest):**

```bash
npm run web
```

Then open <http://localhost:3000> in your browser.

**MCP Server (for Claude Desktop):**

```bash
npm start
```

See [docs/MCP_CLIENT_CONFIG.md](docs/MCP_CLIENT_CONFIG.md) for Claude Desktop setup.

### πŸŽ‰ You're Ready!

Try asking questions like:
- "How do I install K3s on SUSE?"
- "What are the differences between K3s and RKE2?"
- "Show me Rancher backup procedures"

See [docs/EXAMPLES.md](docs/EXAMPLES.md) for more usage examples.

## πŸ› οΈ Available Tools

The MCP server provides these tools:

### `search_docs`
Search documentation using semantic search.
```json
{
  "query": "How do I install K3s on SUSE?",
  "source": "all",
  "limit": 5
}
```

### `ask_question`
Ask questions about documentation and get AI-generated answers with sources.
```json
{
  "question": "What are the differences between K3s and RKE2?",
  "context": "deployment on SUSE Linux Enterprise"
}
```

### `summarize_doc`
Generate AI summaries of documentation pages.
```json
{
  "url": "https://docs.k3s.io/installation",
  "format": "bullet-points"
}
```

### `get_doc_section`
Retrieve specific documentation content.
```json
{
  "url": "https://documentation.suse.com/sles/15-SP5/"
}
```

### `index_documentation`
Index documentation for faster searching.
```json
{
  "source": "k3s",
  "forceRefresh": false
}
```

### `list_doc_sources`
View all available documentation sources and their status.

## πŸ“– Usage Examples

### Web Interface (Easiest!)

Start the web interface and access it from your browser:

```bash
npm run web
```

Then open **http://localhost:3000** in your browser. See [docs/WEB_GUI.md](docs/WEB_GUI.md) for details.

### With Claude Desktop

1. Configure Claude Desktop (see [docs/INSTALL.md](docs/INSTALL.md))
2. Ask Claude to use the tools:

```
"Can you search the SUSE documentation for information about container security?"

"Use the docs navigator to find K3s installation instructions"

"Summarize the Rancher high availability setup documentation"
```

### Direct MCP Usage

```bash
# Start the MCP server
npm start

# The server communicates via stdio using MCP protocol
```

### Command-Line Tools

```bash
# Indexing
npm run index [source]      # Index documentation (k3s, rancher, suse, all)
npm run index all           # Index all sources
npm run index k3s --force   # Force refresh (ignore cache)

# Cache Management
npm run stats               # Show cache statistics
npm run analytics           # Comprehensive analytics report
npm run query-cache [opts]  # Query cache with filters
npm run clear-cache         # Clear all caches
npm run validate            # Validate cache integrity
npm run clear-locks         # Clear expired indexing locks

# Utilities
npm run fix-sources         # Fix legacy cache entries
npm run mark-indexed        # Mark documents as indexed
npm run migrate-sqlite      # Migrate JSON cache to SQLite

# Testing
npm test                    # Run test suite
```

## πŸ—οΈ Architecture

```text
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                     User Interfaces                          β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚   Web Browser   β”‚  Claude Desktop β”‚   Direct MCP Client      β”‚
β”‚  (port 3000)    β”‚   (MCP stdio)   β”‚     (stdio/JSON)         β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
         β”‚                 β”‚                 β”‚
         β–Ό                 β–Ό                 β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚              Application Layer (Node.js)                     β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚  web-server.js  β”‚     index.js (MCP)    β”‚   CLI Tools       β”‚
β”‚   (Express)     β”‚    (stdio protocol)   β”‚  (index-docs.js)  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
         β”‚                    β”‚                    β”‚
         β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                              β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                   Service Layer                              β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚  AI Service      β”‚  Vector Service  β”‚  Doc Service         β”‚
β”‚  - Ollama Q&A    β”‚  - Vectra DB     β”‚  - HTTP fetching     β”‚
β”‚  - OpenAI embed  β”‚  - Embeddings    β”‚  - HTML parsing      β”‚
β”‚  - Summarization β”‚  - Similarity    β”‚  - Cache management  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
         β”‚                  β”‚                    β”‚
         β–Ό                  β–Ό                    β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                      Data Layer                              β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ SQLite Cache  β”‚  Vectra Vectors β”‚  Embedding Cache (JSON)   β”‚
β”‚ - Page cache  β”‚  - Doc chunks   β”‚  - Text hash β†’ vector     β”‚
β”‚ - Metadata    β”‚  - Embeddings   β”‚  - Fast deduplication     β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
```

### Key Components

- **MCP Server** (`index.js`): Implements Model Context Protocol for tool execution via stdio
- **Web Server** (`web-server.js`): Express server providing browser-based UI
- **AI Service**: Handles LLM interactions for Q&A and summarization (Ollama/OpenAI/Anthropic)
- **Cache Service**: SQLite-based page caching with advanced queries and analytics
- **Vector Service**: Manages semantic search using Vectra vector database
- **Documentation Service**: Fetches, parses, and indexes documentation with smart caching

### Data Flow

1. **Indexing**: Docs β†’ Fetch β†’ Parse β†’ Chunk β†’ Embed β†’ Store in Vectra + Cache
2. **Searching**: Query β†’ Embed β†’ Vector Search β†’ Retrieve Docs β†’ Return Results
3. **Q&A**: Question β†’ Context Search β†’ LLM β†’ Answer with Citations

## πŸ”§ Configuration

Edit `.env` to configure:

```env
# AI Provider (ollama, openai, anthropic)
AI_PROVIDER=ollama

# Ollama Settings
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=llama3.2:latest
EMBEDDING_MODEL=nomic-embed-text

# Documentation Sources
SUSE_DOCS_BASE_URL=https://documentation.suse.com
RANCHER_DOCS_URL=https://ranchermanager.docs.rancher.com
K3S_DOCS_URL=https://docs.k3s.io

# Vector Database
VECTOR_DB_PATH=./data/vectors
```

## πŸ“ Project Structure

```text
Docs-Navigator-MCP-SUSE-Edition/
β”œβ”€β”€ docs/              # Documentation files
β”‚   β”œβ”€β”€ ARCHITECTURE.md         # System design and components
β”‚   β”œβ”€β”€ INSTALL.md              # Detailed installation guide
β”‚   β”œβ”€β”€ QUICKREF.md             # Command quick reference
β”‚   β”œβ”€β”€ EXAMPLES.md             # Usage examples
β”‚   β”œβ”€β”€ WEB_GUI.md              # Web interface guide
β”‚   β”œβ”€β”€ MCP_CLIENT_CONFIG.md    # MCP client configuration
β”‚   β”œβ”€β”€ SQLITE_MIGRATION.md     # SQLite caching guide
β”‚   └── CONTRIBUTING.md         # Contribution guidelines
β”‚
β”œβ”€β”€ scripts/           # Setup and utility scripts
β”‚   β”œβ”€β”€ setup.sh       # Linux/macOS setup script
β”‚   └── setup.bat      # Windows setup script
β”‚
β”œβ”€β”€ src/               # Source code (organized by purpose)
β”‚   β”œβ”€β”€ cli/           # Command-line tools
β”‚   β”‚   β”œβ”€β”€ index-docs.js       # Main indexing CLI
β”‚   β”‚   β”œβ”€β”€ cache-analytics.js  # Analytics reports
β”‚   β”‚   β”œβ”€β”€ query-cache.js      # Cache queries
β”‚   β”‚   └── clear-locks.js      # Lock management
β”‚   β”‚
β”‚   β”œβ”€β”€ services/      # Core business logic
β”‚   β”‚   β”œβ”€β”€ ai-service.js           # AI/LLM integration
β”‚   β”‚   β”œβ”€β”€ cache-service.js        # SQLite cache management
β”‚   β”‚   β”œβ”€β”€ documentation-service.js # Doc fetching & parsing
β”‚   β”‚   └── vector-service.js       # Vector database ops
β”‚   β”‚
β”‚   β”œβ”€β”€ tests/         # Test scripts
β”‚   β”‚   β”œβ”€β”€ test.js                 # Main test suite
β”‚   β”‚   β”œβ”€β”€ test-cache.js           # Cache tests
β”‚   β”‚   β”œβ”€β”€ test-concurrent-locks.js # Lock tests
β”‚   β”‚   └── test-ollama.js          # Ollama integration tests
β”‚   β”‚
β”‚   β”œβ”€β”€ utils/         # Maintenance utilities
β”‚   β”‚   β”œβ”€β”€ migrate-to-sqlite.js    # JSONβ†’SQLite migration
β”‚   β”‚   β”œβ”€β”€ fix-cache-sources.js    # Cache repair tools
β”‚   β”‚   └── mark-indexed.js         # Status updates
β”‚   β”‚
β”‚   β”œβ”€β”€ index.js       # MCP server entry point
β”‚   └── web-server.js  # Web UI server
β”‚
β”œβ”€β”€ public/            # Web GUI assets (HTML, CSS, JS)
β”œβ”€β”€ data/              # Data storage
β”‚   β”œβ”€β”€ vectors/       # Vector database (Vectra)
β”‚   β”œβ”€β”€ page-cache.db  # SQLite page cache
β”‚   β”œβ”€β”€ embedding-cache.json # Embedding cache
β”‚   └── html/          # Cached HTML files
β”‚
└── .env               # Environment configuration
```

## πŸ”§ Troubleshooting

### Installation Issues

**Problem: `npm install` fails**

```bash
# Clear npm cache and retry
npm cache clean --force
rm -rf node_modules package-lock.json
npm install
```

**Problem: Node.js version too old**

```bash
# Check version (needs 18+)
node --version

# Update Node.js from https://nodejs.org/
# Or use nvm:
nvm install 18
nvm use 18
```

### Ollama Issues

**Problem: "Connection refused" or "ECONNREFUSED"**

```bash
# Check if Ollama is running
curl http://localhost:11434/api/tags

# Start Ollama
ollama serve

# Or check Ollama is installed
ollama --version
```

**Problem: Models not found**

```bash
# List installed models
ollama list

# Pull required models
ollama pull llama3.2:latest
ollama pull nomic-embed-text

# Verify models work
ollama run llama3.2:latest "Hello"
```

**Problem: Ollama too slow**

```bash
# Use OpenAI for embeddings instead
# Edit .env:
EMBEDDING_PROVIDER=openai
OPENAI_API_KEY=your_key_here

# Keep Ollama for Q&A (free and private)
AI_PROVIDER=ollama
```

### Indexing Issues

**Problem: Indexing fails with "Item already exists"**

```bash
# This happens with parallel indexing - use sequential mode
# In .env file:
FETCH_BATCH_SIZE=1

# Or clear vectors and reindex
npm run clear-cache
npm run index all
```

**Problem: "Another indexing process is already running"**

```bash
# Clear stale locks
npm run clear-locks

# Then retry indexing
npm run index all
```

**Problem: Indexing very slow**

```bash
# Use OpenAI embeddings (much faster than Ollama)
# Edit .env:
EMBEDDING_PROVIDER=openai
OPENAI_API_KEY=your_key_here

# Ollama embeddings take ~30 min for 110 docs
# OpenAI embeddings take ~2 min for same docs
```

**Problem: "404 Not Found" during indexing**

```bash
# Some documentation URLs may have changed
# Check cache analytics for details
npm run analytics

# Rebuild specific source
npm run rebuild suse
```

### Cache Issues

**Problem: UI shows "0 documents indexed"**

```bash
# Check actual cache status
npm run stats

# If cache exists but UI shows 0, restart web server
npm run web

# If genuinely empty, index documentation
npm run index all
```

**Problem: Outdated cached content**

```bash
# Force refresh all caches
npm run index all --force

# Or clear and reindex
npm run clear-cache
npm run index all
```

**Problem: Cache corruption**

```bash
# Validate cache integrity
npm run validate

# If errors found, rebuild
npm run clear-cache
npm run migrate-sqlite  # If migrating from JSON
npm run index all
```

### Web Interface Issues

**Problem: Web server won't start**

```bash
# Check if port 3000 is in use
lsof -i :3000  # Linux/Mac
netstat -ano | findstr :3000  # Windows

# Kill process using port 3000 or change port
# Edit src/web-server.js: const PORT = 3001;
```

**Problem: "No results found" in web UI**

```bash
# Verify documents are indexed
npm run stats

# Check AI service is working
npm test

# Verify Ollama/OpenAI connection
curl http://localhost:11434/api/tags  # Ollama
```

### MCP Server Issues

**Problem: Claude Desktop can't connect**

```bash
# Verify MCP server starts
npm start

# Check Claude Desktop config file location:
# macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
# Windows: %APPDATA%\Claude\claude_desktop_config.json

# Validate config syntax (must be valid JSON)
# See docs/MCP_CLIENT_CONFIG.md for examples
```

**Problem: Tools not appearing in Claude**

```bash
# Restart Claude Desktop completely
# Check server logs for errors
npm start 2>&1 | tee server.log
```

### Performance Issues

**Problem: Slow searches**

```bash
# Check cache stats
npm run analytics

# Ensure documents are indexed
npm run stats

# Try rebuilding vector index
npm run clear-cache vectors
npm run index all
```

**Problem: High memory usage**

```bash
# Reduce batch sizes in .env:
FETCH_BATCH_SIZE=1
EMBEDDING_CONCURRENCY=1

# Or use OpenAI embeddings (more efficient)
EMBEDDING_PROVIDER=openai
```

### Database Issues

**Problem: SQLite database locked**

```bash
# Close all processes accessing database
pkill -f "node src"

# Clear locks
npm run clear-locks

# If persists, delete and rebuild
rm data/page-cache.db
npm run index all
```

**Problem: Want to revert to JSON cache**

```bash
# Edit .env file:
USE_JSON_CACHE=true
PAGE_CACHE_PATH=./data/page-cache.json

# Restart services
npm run web
```

### Getting Help

1. **Check logs**: Look for error messages in terminal output
2. **Run diagnostics**: `npm run stats` and `npm run analytics`
3. **Validate setup**: `npm test` to run test suite
4. **Check documentation**: See `docs/` folder for detailed guides
5. **GitHub Issues**: Report bugs at repository issues page

### Common Error Messages

| Error | Solution |
|-------|----------|
| `ECONNREFUSED` | Ollama not running - start with `ollama serve` |
| `Item already exists` | Use `FETCH_BATCH_SIZE=1` in .env |
| `Lock held by another process` | Run `npm run clear-locks` |
| `No such table: locks` | Normal on first run - table created automatically |
| `404 Not Found` | Documentation URL changed - run `npm run rebuild` |
| `ENOENT: no such file` | Create data directories: `mkdir -p data/{vectors,html}` |

## 🀝 Contributing

Contributions welcome! This project started during **Hack Week 25**.

See [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md) for guidelines.

## πŸ“„ License

See [LICENSE](LICENSE) file for details.

## 🎯 Use Cases

- **DevOps Engineers**: Quickly find deployment and configuration info
- **System Administrators**: Navigate SUSE Linux documentation efficiently
- **Kubernetes Users**: Get instant answers about K3s and Rancher
- **Technical Writers**: Research and cross-reference documentation
- **Support Teams**: Find solutions faster with semantic search

## πŸ”— Resources

### Platform & Tools
- [Model Context Protocol](https://modelcontextprotocol.io)
- [Ollama](https://ollama.ai)

### Documentation Sources
- [SUSE Documentation](https://documentation.suse.com)
- [Rancher Documentation](https://ranchermanager.docs.rancher.com)
- [K3s Documentation](https://docs.k3s.io)
- [RKE2 Documentation](https://docs.rke2.io)
- [Longhorn Documentation](https://longhorn.io/docs)
- [Harvester Documentation](https://docs.harvesterhci.io)
- [NeuVector Documentation](https://open-docs.neuvector.com)
- [Kubewarden Documentation](https://docs.kubewarden.io)

---

Built with ❀️ for Hack Week 25.

TDQS

A3.9/5.0

Scored across 6 tools

Disambiguation5/5

Each tool has a clear, distinct purpose: direct retrieval, semantic search, summarization, Q&A, index management, and source listing. No two tools overlap in function, and the descriptions clearly differentiate them.

Naming Consistency5/5

All tool names follow the verb_noun pattern with lowercase and underscores. The verbs (get, search, summarize, ask, index, list) are consistent and the nouns are descriptive. Minor singular/plural variations do not detract from the overall pattern.

Tool Count5/5

The server has 6 tools, which is well within the ideal range. Each tool serves a necessary function for a documentation navigator, with no redundant or superfluous tools.

Completeness5/5

The tool surface covers the core workflows of a documentation navigator: indexing sources, searching, retrieving specific sections, summarizing, and asking questions. There are no obvious dead ends or missing operations for the stated purpose.

Maintenance

ActivityInactive
ResponsivenessNo issues