# 🧠 ontology-rag-mcp

> A generic **code ontology** platform that ingests Spring Boot REST microservice codebases from GitHub, indexes them into Apache Solr, builds a REST request-flow ontology + relationship graph, and serves it through a **Model Context Protocol (MCP)** server — so any MCP client (Cursor, Claude, etc.) can ask natural-language questions about the code and get accurate, flow-aware answers with citations.

[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/downloads/)
[![Solr 9.x](https://img.shields.io/badge/Solr-9.x-orange.svg)](https://solr.apache.org/)
[![MCP Protocol](https://img.shields.io/badge/protocol-MCP-green.svg)](https://modelcontextprotocol.io/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

---

## 📋 Table of Contents

- [Overview](#-overview)
- [Key Features](#-key-features)
- [Hard Constraints](#-hard-constraints)
- [Architecture](#-architecture)
  - [Local Path (git + docker)](#local-path-git--docker)
  - [Headless Path (GitHub → Jenkins → SSH)](#headless-path-github--jenkins--ssh)
- [Prerequisites](#-prerequisites)
- [Quickstart](#-quickstart)
  - [Option 1: Docker Compose (Recommended)](#option-1-docker-compose-recommended)
  - [Option 2: Local Development](#option-2-local-development)
- [Configuring Your AI IDE](#-configuring-your-ai-ide)
  - [Cursor](#1--cursor)
  - [Claude Desktop](#2--claude-desktop)
  - [VS Code with Continue / Cline](#3--vs-code-with-continue--cline)
- [How It Works](#-how-it-works)
  - [Ingestion Pipeline](#ingestion-pipeline)
  - [REST Flow Ontology](#rest-flow-ontology)
  - [Hybrid Retrieval](#hybrid-retrieval)
  - [Incremental Indexing](#incremental-indexing)
- [CLI Reference](#-cli-reference)
- [Environment Variables](#-environment-variables)
- [Complete MCP Tool Reference](#-complete-mcp-tool-reference)
- [Pluggable Providers](#-pluggable-providers)
- [Sample Queries](#-sample-queries)
- [Project Structure](#-project-structure)
- [Tech Choices](#-tech-choices)
- [Development & Testing](#-development--testing)
- [Troubleshooting](#-troubleshooting)
- [Author & Contact](#-author--contact)
- [License](#-license)

---

## 🌐 Overview

**ontology-rag-mcp** turns a GitHub repository into a queryable **knowledge layer** — not just semantic search, but an understanding of:

- REST endpoints (`@GetMapping`, `@PostMapping`, etc.)
- Request flows behind each endpoint (Controller → Service → Repository)
- Service relationships and cross-service calls (`@FeignClient`, `RestTemplate`, `WebClient`)
- Configuration files, README/docs, and OpenAPI specs linked to the code they describe

### How is this different from plain RAG?

| Plain RAG | ontology-rag-mcp |
|---|---|
| Chunks code by text similarity | Builds a **request-flow ontology** per endpoint |
| Returns similar-looking snippets | Returns **ordered call chains** with Mermaid diagrams |
| No graph awareness | Persists `callsOut` / `calledBy` edges in Solr fields |
| Needs an LLM to answer | Works **fully offline** with retrieval-only mode |
| Tied to one vector DB | Uses **only Solr 9** (BM25 + dense vector kNN) |

The whole point: any MCP client can ask *"walk me through what happens when a user places an order"* and get the real Controller→Service→Repository chain — even when the query names no class or method.

---

## ✨ Key Features

| Feature | Description |
|---|---|
| **REST Flow Ontology** | Per-endpoint call graph: Controller → Service → Repository → external calls |
| **Hybrid Retrieval** | Solr BM25 + dense vector kNN fused with Reciprocal Rank Fusion (RRF) |
| **Flow-Aware Search** | Intent routing + flow-doc seeding for vague "how does X work" questions |
| **13 MCP Tools** | Typed DTOs — `rag_search`, `correct_answer`, `flow_of`, `find_rest_endpoints`, and more |
| **Offline Embeddings** | `BAAI/bge-small-en-v1.5` (384-dim) via sentence-transformers — no paid API |
| **Optional LLM** | OpenAI-compatible endpoint for narrative summaries (Ollama, Azure, vLLM, etc.) |
| **Incremental Indexing** | Commit SHA tracking — re-runs only re-index changed files |
| **Pluggable Providers** | Local defaults + optional GitHub/Jenkins/SSH MCP adapters |
| **Docker Compose** | `docker compose up` starts Solr 9 + MCP server in one command |
| **Spring Boot First** | Parses `@RestController`, `@Service`, `@Repository`, `@FeignClient` |

---

## 🔒 Hard Constraints

These are architectural invariants — the platform is designed around them:

| Constraint | Implementation |
|---|---|
| **Retrieval stack** | Apache Solr 9.x only (BM25 + dense vector kNN). No Pinecone/Weaviate/Chroma/Neo4j. Graph is in-memory, persisted as Solr fields. |
| **Source of code** | Git/GitHub only (public repos or private via `GITHUB_TOKEN`). No Perforce/Bitbucket/NAS. |
| **Target codebases** | REST Spring Boot microservices (Java, Maven/Gradle). Parser is pluggable for future languages. |
| **Embeddings** | Local/offline by default (`sentence-transformers`). No paid embedding API required. |
| **LLM** | Optional. System fully functions with `LLM_ENABLED=false`. |
| **Configuration** | Everything via environment variables. Secrets never committed. |
| **Core reproducibility** | Any developer can run with just `git` + `docker compose` — MCP adapters are optional. |

---

## 🏗 Architecture

### Local Path (git + docker)

The default path — no external MCP servers required. Fully reproducible from a public GitHub clone.

```mermaid
flowchart TB
    subgraph source [Source]
        GitHub[GitHub Repo] --> GitClone[GitPython Shallow Clone]
        ReposYml[repos.yml] --> GitClone
    end

    subgraph parse [Parse & Ontology]
        GitClone --> JavaParser[javalang Java Parser]
        JavaParser --> ClassDocs[class / method / endpoint docs]
        JavaParser --> FlowBuilder[Flow Ontology Builder]
        FlowBuilder --> FlowDocs[flow docs + Mermaid]
        FlowBuilder --> GraphEdges[callsOut / calledBy edges]
    end

    subgraph solr [Solr 9]
        ClassDocs --> RawCol[product-raw collection]
        FlowDocs --> RawCol
        GraphEdges --> RawCol
        RawCol --> Embed[Local Embeddings bge-small-en-v1.5]
        Embed --> RAGCol[product-rag collection]
    end

    subgraph serve [Serving]
        RAGCol --> Retrieval[Hybrid Retrieval BM25 + kNN + RRF]
        Retrieval --> Intent[Intent Classifier]
        Intent --> MCP[FastMCP Server]
        MCP --> Client[Cursor / Claude / any MCP client]
    end
```

```
┌─────────────────────────────────────────────────────────────────────┐
│                     AI IDE (MCP Client)                             │
│              Cursor / Claude Desktop / VS Code                      │
└──────────────────────────┬──────────────────────────────────────────┘
                           │  HTTP (streamable-http) or stdio
┌──────────────────────────▼──────────────────────────────────────────┐
│                  MCP Server (ontology-rag serve)                  │
│                    13 tools · FastMCP + Python                      │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────────┐  │
│  │  Retriever   │  │  Intent      │  │  LLM Client (optional)   │  │
│  │  BM25+kNN    │  │  Classifier  │  │  OpenAI-compatible       │  │
│  └──────┬───────┘  └──────────────┘  └──────────────────────────┘  │
└─────────┼───────────────────────────────────────────────────────────┘
          │
┌─────────▼───────────────────────────────────────────────────────────┐
│                    Apache Solr 9.x                                  │
│  ┌─────────────────────┐    ┌─────────────────────────────────┐    │
│  │  {product}-raw      │───▶│  {product}-rag                  │    │
│  │  (source of truth)  │    │  (+ 384-dim embedding vectors)  │    │
│  └─────────────────────┘    └─────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────────────┘
```

### Headless Path (GitHub → Jenkins → SSH)

Optional adapters for zero-manual-step deployment. The **core platform does not depend on these** — they degrade to local defaults when absent.

```mermaid
sequenceDiagram
    participant Agent as Cursor Agent
    participant Jenkins as Jenkins MCP
    participant Platform as ontology-rag-mcp
    participant GitHub as GitHub MCP
    participant SSH as Linux SSH MCP
    participant Target as Target Linux Host

    Agent->>Jenkins: trigger ontology-rag-onboard
    Note over Jenkins: Params: REPOS, BRANCH, PRODUCT, TARGET_HOST

    Jenkins->>Platform: checkout-platform
    Platform->>GitHub: fetch source + record commit SHA
    GitHub-->>Platform: file tree + metadata

    Platform->>Platform: parse + index + build flow ontology
    Platform->>Platform: embed into Solr

    Jenkins->>SSH: deploy-mcp to TARGET_HOST
    SSH->>Target: sync code, create venv, start MCP on free port
    SSH->>Target: health-check + tail logs

    Jenkins-->>Agent: {"petclinic-ontology": {"url": "http://host:port/mcp"}}
```

| Stage | What Happens |
|---|---|
| `checkout-platform` | Clone ontology-rag-mcp, install deps |
| `fetch-source` | GitHub MCP (or GitPython) fetches repos, records SHA |
| `parse+index` | Java parser → Solr raw collection |
| `build-flow-ontology` | Endpoint extraction, call graph, flow docs |
| `embed` | Local embeddings → Solr serving collection |
| `deploy-mcp` | SSH MCP starts MCP server on target host |
| `verify` | Health-check + print `mcp.json` snippet |

---

## 📦 Prerequisites

### For Docker Compose (recommended)

| Requirement | Version | Verify |
|---|---|---|
| **Docker** | 20.10+ | `docker --version` |
| **Docker Compose** | v2+ | `docker compose version` |
| **Git** | any | `git --version` |

### For local development

| Requirement | Version | Verify |
|---|---|---|
| **Python** | 3.11+ | `python --version` |
| **Git** | any | `git --version` |
| **Apache Solr 9** | 9.x | via Docker, or `curl http://localhost:8983/solr/admin/info/system` |

> **First ingest downloads ~130 MB** for the embedding model (`BAAI/bge-small-en-v1.5`). Subsequent runs use the cached model.

---

## 🚀 Quickstart

### Option 1: Docker Compose (Recommended)

```powershell
# 1. Clone and configure
cd C:\AI_Workspaces\Anti_Workspace\ontology-rag-mcp
copy .env.example .env

# 2. Start Solr + MCP server
docker compose up -d

# 3. Wait for Solr to be healthy (~30s), then ingest the sample repo
docker compose exec app ontology-rag ingest

# 4. One-shot RAG query
docker compose exec app ontology-rag ask "What REST endpoints does this expose?"

# 5. MCP server is already running — check startup logs for mcp.json snippet
docker compose logs app
```

The default sample repo is [spring-projects/spring-petclinic](https://github.com/spring-projects/spring-petclinic) (configured in `repos.yml`).

**Expected ingest output:**

```
=== Ingesting petclinic ===
Cloning https://github.com/spring-projects/spring-petclinic (branch=main)
Indexing 150+ documents into petclinic-raw
Embedding 150+ documents
Ingest complete: {'repos': 1, 'files': 80, 'docs': 150, 'embedded': 150}
```

### Option 2: Local Development

```powershell
cd C:\AI_Workspaces\Anti_Workspace\ontology-rag-mcp

# Create virtual environment
python -m venv .venv
.venv\Scripts\Activate.ps1

# Install
pip install -e ".[dev]"

# Copy config
copy .env.example .env

# Start Solr separately (or use docker compose up solr -d)
docker compose up solr -d

# Ingest
ontology-rag ingest

# Start MCP server
ontology-rag serve
```

On startup, the server prints a ready-to-paste `mcp.json` snippet:

```json
{
  "ontology-rag": {
    "url": "http://localhost:8765/mcp"
  }
}
```

---

## 🔌 Configuring Your AI IDE

ontology-rag-mcp supports two MCP transports:

| Transport | Use Case | Config Style |
|---|---|---|
| `streamable-http` (default) | Docker / remote deploy | URL-based |
| `stdio` | Local dev without HTTP | Command-based |

Set `MCP_TRANSPORT=streamable-http` or `MCP_TRANSPORT=stdio` in `.env`.

---

### 1. 🖱 Cursor

#### Streamable HTTP (Docker / deployed server)

Create `.cursor/mcp.json` in your project (or `~/.cursor/mcp.json` globally):

```json
{
  "mcpServers": {
    "ontology-rag": {
      "url": "http://localhost:8765/mcp"
    }
  }
}
```

> The server prints this snippet on startup when you run `ontology-rag serve`.

#### stdio (local development)

```json
{
  "mcpServers": {
    "ontology-rag": {
      "command": "ontology-rag",
      "args": ["serve"],
      "env": {
        "MCP_TRANSPORT": "stdio",
        "SOLR_BASE_URL": "http://localhost:8983/solr",
        "RAG_PRODUCT": "petclinic"
      }
    }
  }
}
```

#### Verifying in Cursor

1. Open **Settings → MCP** (or `Ctrl+Shift+P` → "MCP")
2. Look for `ontology-rag` with a green status indicator
3. Try: *"What REST endpoints does this expose?"*
4. The agent should call `find_rest_endpoints` and return cited routes

---

### 2. 🤖 Claude Desktop

#### Configuration File Location

| OS | Path |
|---|---|
| **Windows** | `%APPDATA%\Claude\claude_desktop_config.json` |
| **macOS** | `~/Library/Application Support/Claude/claude_desktop_config.json` |

#### Streamable HTTP

```json
{
  "mcpServers": {
    "ontology-rag": {
      "url": "http://localhost:8765/mcp"
    }
  }
}
```

#### stdio

```json
{
  "mcpServers": {
    "ontology-rag": {
      "command": "ontology-rag",
      "args": ["serve"],
      "env": {
        "MCP_TRANSPORT": "stdio",
        "SOLR_BASE_URL": "http://localhost:8983/solr",
        "RAG_PRODUCT": "petclinic"
      }
    }
  }
}
```

Restart Claude Desktop after saving. Look for the tools icon in the chat input.

---

### 3. 💻 VS Code with Continue / Cline

Add to `.continue/config.json` or Cline MCP settings:

```json
{
  "mcpServers": {
    "ontology-rag": {
      "url": "http://localhost:8765/mcp"
    }
  }
}
```

---

## ⚡ How It Works

### Ingestion Pipeline

```
repos.yml / CLI --repos
        │
        ▼
┌───────────────────┐
│  Source Provider  │  SOURCE_PROVIDER=local → GitPython shallow clone
│                   │  SOURCE_PROVIDER=github → GitHub MCP (falls back to git)
└────────┬──────────┘
         │  Records commit SHA for incremental re-runs
         ▼
┌───────────────────┐
│  File Curation    │  Skips: target/, build/, .git/, node_modules/,
│                   │  *.class, *.jar, /test/, generated-sources/
└────────┬──────────┘
         ▼
┌───────────────────┐
│  Java Parser      │  javalang → class-level docs (+ optional method chunks)
│  (javalang)       │  Also: application.yml, README, OpenAPI specs
└────────┬──────────┘
         ▼
┌───────────────────┐
│  Flow Ontology    │  Endpoint extraction, call graph, flow docs + Mermaid
│  Builder          │  Cross-service: @FeignClient, RestTemplate, WebClient
└────────┬──────────┘
         ▼
┌───────────────────┐
│  Solr Raw Index   │  {product}-raw — source of truth, no vectors
└────────┬──────────┘
         ▼
┌───────────────────┐
│  Embed Pipeline   │  BAAI/bge-small-en-v1.5 → {product}-rag collection
│                   │  Optional LLM summaries if LLM_ENABLED=true
└───────────────────┘
```

**Chunk types indexed:**

| `chunkType` | Description |
|---|---|
| `class` | One doc per Java class (package, annotations, methods, dependencies) |
| `method` | Per-method chunk (when `RAG_METHOD_LEVEL=true`) |
| `endpoint` | REST route: HTTP method, path, controller#method, request/response types |
| `flow` | Ordered request flow for an endpoint + Mermaid sequence diagram |
| `config` | `application.yml` / `application.properties` |
| `document` | README, markdown, OpenAPI/Swagger specs |
| `correction` | LLM-verified user correction (runtime re-embed; survives ingest) |

### REST Flow Ontology

For every `@RestController` method with an HTTP mapping, the platform:

1. **Extracts the endpoint** — class-level `@RequestMapping` prefix + method `@GetMapping` etc.
2. **Resolves dependencies** — `@Autowired` fields, constructor injection, `@Qualifier`
3. **Traces the call chain** — Controller → Service(s) → Repository / external call (bounded depth, cycle-safe)
4. **Detects cross-service calls** — `@FeignClient` interfaces, `RestTemplate`/`WebClient` usage
5. **Persists as Solr fields** — `callsOut`, `calledBy`, `flowName`, `httpMethod`, `route`
6. **Links docs to code** — README/OpenAPI chunks inherit flows from mentioned class/endpoint names

Example flow doc for `POST /api/orders`:

```
1. [controller] com.example.OrderController#createOrder
2. [service]    com.example.OrderService#createOrder
3. [repository] com.example.OrderRepository#save
```

Plus a Mermaid sequence diagram:

```mermaid
sequenceDiagram
    participant Client
    participant OrderController as OrderController
    participant OrderService as OrderService
    participant OrderRepository as OrderRepository
    Client->>OrderController: POST /api/orders
    OrderController->>OrderService: createOrder
    OrderService->>OrderRepository: save
```

### Hybrid Retrieval

```
User query
    │
    ▼
┌─────────────────────┐
│  Intent Classifier  │  regex-based (no LLM): flow / endpoint / class / config / default
└────────┬────────────┘
         ▼
┌─────────────────────┐     ┌─────────────────────┐
│  Solr BM25 (edismax)│     │  Solr kNN (384-dim) │
│  field boosts per   │     │  cosine similarity  │
│  intent profile     │     │                     │
└────────┬────────────┘     └────────┬────────────┘
         │                           │
         └───────────┬───────────────┘
                     ▼
         ┌───────────────────────┐
         │  RRF Fusion           │
         └───────────┬───────────┘
                     ▼
         ┌───────────────────────┐
         │  Quality Levers       │
         │  · down-rank getters│
         │  · MMR diversity    │
         │  · adaptive rerank  │
         │  · flow-doc seeding │
         │  · call-graph fusion│
         └───────────┬───────────┘
                     ▼
              Shaped results with citations
              (file path + line range + FQN)
```

**Flow-doc seeding** (important): when intent is `flow` and no `flow`/`endpoint` doc is in the top results, the retriever runs an extra chunkType-restricted kNN and injects the best-matching flow doc — so anchorless questions like *"walk me through what happens when a user places an order"* surface the real endpoint flow.

### Incremental Indexing

Each repo's latest commit SHA is persisted in `.ingest-cache/commit_shas.json`.

| Scenario | Behavior |
|---|---|
| Re-run with **no new commits** | SHA unchanged → incremental no-op for changed files |
| Re-run after **new commit** | Only changed files (added/modified/deleted) are re-indexed |
| **Full rebuild** | Delete `.ingest-cache/` or set `REBUILD=full` in Jenkins |

---

## 🖥 CLI Reference

```bash
ontology-rag ingest [--repos URL1,URL2] [--branch main]
ontology-rag serve
ontology-rag ask "your question here"
```

| Command | Description |
|---|---|
| `ingest` | Fetch → parse → index → build flow ontology → embed. Reads `repos.yml` by default. |
| `serve` | Start MCP server (streamable HTTP or stdio). Prints `mcp.json` snippet. |
| `ask` | One-shot hybrid RAG query with citations (demo / debugging). |

**Examples:**

```powershell
# Ingest a specific repo
ontology-rag ingest --repos https://github.com/spring-projects/spring-petclinic --branch main

# Ingest multiple repos (microservices)
ontology-rag ingest --repos https://github.com/org/order-service,https://github.com/org/payment-service

# Ask a flow question
ontology-rag ask "Walk me through what happens when a user creates an order"

# Ask about cross-service calls
ontology-rag ask "Which service calls the payment service?"
```

---

## 🔐 Environment Variables

All configuration is via environment variables. Copy `.env.example` to `.env` and adjust.

### Provider Selection

| Variable | Required | Default | Description |
|:---|:---:|:---:|:---|
| `SOURCE_PROVIDER` | No | `local` | `local` = GitPython clone. `github` = GitHub MCP adapter (falls back to local). |
| `ORCHESTRATOR` | No | `local` | `local` = CLI. `jenkins` = Jenkins MCP orchestrator. |
| `DEPLOY_PROVIDER` | No | `docker` | `docker` = docker compose. `ssh` = Linux SSH MCP deploy. |

### Solr

| Variable | Required | Default | Description |
|:---|:---:|:---:|:---|
| `SOLR_BASE_URL` | No | `http://localhost:8983/solr` | Solr base URL (no trailing collection name). |
| `SOLR_PORT` | No | `8983` | Host port for Solr container. |
| `RAG_PRODUCT` | No | `petclinic` | Collection prefix. Creates `{product}-raw` and `{product}-rag`. |
| `RAG_COLLECTION` | No | *(auto)* | Override serving collection name. Defaults to `{RAG_PRODUCT}-rag`. |

### Embeddings

| Variable | Required | Default | Description |
|:---|:---:|:---:|:---|
| `RAG_EMBED_MODEL` | No | `BAAI/bge-small-en-v1.5` | Local sentence-transformers model. |
| `RAG_EMBED_DIM` | No | `384` | Vector dimension (must match model). |
| `RAG_RERANK_ENABLED` | No | `false` | Enable cross-encoder reranking (`BAAI/bge-reranker-base`). |
| `RAG_RERANK_MODEL` | No | `BAAI/bge-reranker-base` | Reranker model name. |

### Retrieval Feature Flags

| Variable | Required | Default | Description |
|:---|:---:|:---:|:---|
| `RAG_METHOD_LEVEL` | No | `false` | Also index per-method chunks (in addition to class-level). |
| `RAG_FLOW_SEED` | No | `true` | Inject flow/endpoint doc for vague flow questions. |
| `RAG_MMR_ENABLED` | No | `true` | Maximal Marginal Relevance diversity (cap chunks per class). |
| `RAG_SKIP_PATTERNS` | No | `target/,build/,...` | Comma-separated file/path patterns to skip during ingest. |

### LLM (Optional)

| Variable | Required | Default | Description |
|:---|:---:|:---:|:---|
| `LLM_ENABLED` | No | `false` | Enable LLM for answer synthesis and domain summaries. |
| `LLM_BASE_URL` | No | `http://localhost:11434/v1` | OpenAI-compatible API base URL. |
| `LLM_MODEL` | No | `llama3` | Model name for chat completions. |
| `LLM_API_KEY` | No | — | API key (OpenAI, Azure, etc.). Not needed for Ollama. |

> **The system fully functions with `LLM_ENABLED=false`.** Retrieval, MCP tools, and CLI `ask` all work without an LLM. Enabling an LLM adds narrative summaries on top.

### MCP Server

| Variable | Required | Default | Description |
|:---|:---:|:---:|:---|
| `MCP_HOST` | No | `0.0.0.0` | Bind address for HTTP transport. |
| `MCP_PORT` | No | `8765` | Port for streamable HTTP transport. |
| `MCP_TRANSPORT` | No | `streamable-http` | `streamable-http` or `stdio`. |

### Ingestion

| Variable | Required | Default | Description |
|:---|:---:|:---:|:---|
| `INGEST_CACHE_DIR` | No | `./.ingest-cache` | Git clone cache + commit SHA store. |
| `REPOS_CONFIG` | No | `repos.yml` | Path to repos configuration file. |
| `GITHUB_TOKEN` | No | — | Enables private repo access. Never commit this value. |

### SSH Deploy (`DEPLOY_PROVIDER=ssh`)

| Variable | Required | Default | Description |
|:---|:---:|:---:|:---|
| `SSH_HOST` | Yes* | — | Target Linux host for remote MCP deploy. |
| `SSH_USERNAME` | Yes* | — | SSH username. |
| `SSH_PASSWORD` | No | — | SSH password (or use key). |
| `SSH_KEY_PATH` | No | — | Path to SSH private key. |
| `SSH_PORT` | No | `22` | SSH port. |

### Jenkins Orchestrator (`ORCHESTRATOR=jenkins`)

| Variable | Required | Default | Description |
|:---|:---:|:---:|:---|
| `JENKINS_URL` | Yes* | — | Jenkins server URL. |
| `JENKINS_USER` | Yes* | — | Jenkins username. |
| `JENKINS_TOKEN` | Yes* | — | Jenkins API token. |
| `JENKINS_JOB_NAME` | No | `ontology-rag-onboard` | Pipeline job name. |

---

## 📖 Complete MCP Tool Reference

All tools return typed, clean DTOs — not raw Solr fragments. Every result includes citations (file path, line range, FQN) where available.

---

### 🔧 `rag_search`

**Hybrid semantic + keyword search with citations and optional LLM synthesis.**

| Parameter | Type | Required | Default | Description |
|:---|:---:|:---:|:---:|:---|
| `query` | `str` | Yes | — | Natural-language search query. |
| `k` | `int` | No | `10` | Number of results to return. |
| `mode` | `str` | No | `"hybrid"` | `"hybrid"` (BM25 + kNN), `"keyword"` (BM25 only), or `"semantic"` (kNN only). |

**Returns:** `query`, `intent`, `total`, `hits[]` (with citations), `context_block`, `answer` (LLM synthesis if enabled, else context). Verified `correction` hits are ranked first when they match the question.

**Examples:**

```
"Explain the OrderService class"
"How is authentication configured?"
"walk me through what happens when a user places an order"
```

---

### 🔧 `correct_answer`

**When `rag_search` answers incorrectly, submit the right answer. An LLM must verify the correction before it is re-embedded into Solr.**

Requires `LLM_ENABLED=true`. Rejected corrections are not stored.

| Parameter | Type | Required | Default | Description |
|:---|:---:|:---:|:---:|:---|
| `question` | `str` | Yes | — | The original user question. |
| `correct_answer` | `str` | Yes | — | The corrected answer to store. |
| `previous_answer` | `str` | No | `""` | The wrong answer that was given (helps the verifier). |
| `source_hit_id` | `str` | No | `""` | Optional Solr hit `id` from `rag_search` that was misleading. |

**Returns:** `stored`, `verified`, `id`, `reason`, `canonical_answer`.

**Examples:**

```
correct_answer(
  question="What happens when a user creates an order?",
  previous_answer="It only hits the repository.",
  correct_answer="POST /api/orders goes OrderController → OrderService → PaymentClient then OrderRepository.save.",
  source_hit_id="flow-OrderController-createOrder"
)
```

After a successful store, ask the same question again with `rag_search` — the correction is injected and re-used.

---

### 🔧 `list_corrections`

**List verified user corrections currently in the serving index.**

**Returns:** List of `{id, question, canonical_answer, source_hit_id}`.

---

### 🔧 `delete_correction`

**Remove a stored correction.**

| Parameter | Type | Required | Default | Description |
|:---|:---:|:---:|:---:|:---|
| `correction_id` | `str` | Yes | — | Id returned by `correct_answer` (`correction-...`). |

---

### 🔧 `find_rest_endpoints`

**List REST endpoints with method, route, controller, and flow name.**

| Parameter | Type | Required | Default | Description |
|:---|:---:|:---:|:---:|:---|
| `query` | `str` | No | `""` | Free-text search within endpoints. |
| `http_method` | `str` | No | `""` | Filter by HTTP method (e.g., `"GET"`, `"POST"`). |
| `path_contains` | `str` | No | `""` | Filter routes containing this substring. |

**Returns:** List of `EndpointInfo` — `http_method`, `route`, `controller`, `method_name`, `flow_name`, `module`.

**Examples:**

```
find_rest_endpoints()
find_rest_endpoints(http_method="POST")
find_rest_endpoints(path_contains="/orders")
```

---

### 🔧 `flow_of`

**Return the ordered request flow for an endpoint or class, with a Mermaid sequence diagram.**

| Parameter | Type | Required | Default | Description |
|:---|:---:|:---:|:---:|:---|
| `endpoint_or_class` | `str` | Yes | — | Route (e.g., `"/api/orders"`), class name, or FQN. |

**Returns:** `FlowResult` — `name`, `endpoint`, `http_method`, `steps[]` (ordered layers), `mermaid` (diagram string), `cross_service[]`.

**Examples:**

```
flow_of("/api/orders")
flow_of("OrderController")
flow_of("POST /api/orders")
```

---

### 🔧 `callers_of`

**Inbound call edges — who calls this class or method.**

| Parameter | Type | Required | Default | Description |
|:---|:---:|:---:|:---:|:---|
| `class_or_method` | `str` | Yes | — | Class name, FQN, or `ClassName#methodName`. |

**Returns:** List of caller symbol strings.

**Example:**

```
callers_of("PaymentService")
callers_of("OrderService#createOrder")
```

---

### 🔧 `uses_of`

**Outbound call edges — what this class or method calls.**

| Parameter | Type | Required | Default | Description |
|:---|:---:|:---:|:---:|:---|
| `class_or_method` | `str` | Yes | — | Class name, FQN, or `ClassName#methodName`. |

**Returns:** List of callee symbol strings (includes Feign/RestTemplate targets).

**Example:**

```
uses_of("OrderController")
uses_of("PaymentService#processPayment")
```

---

### 🔧 `find_services`

**List `@Service` beans with summaries. Optional query filter.**

| Parameter | Type | Required | Default | Description |
|:---|:---:|:---:|:---:|:---|
| `query` | `str` | No | `""` | Filter services by name or description. |

**Returns:** List of service class docs with annotations, dependencies, and summaries.

---

### 🔧 `get_class`

**Fetch a specific Java class by simple name or fully-qualified name.**

| Parameter | Type | Required | Default | Description |
|:---|:---:|:---:|:---:|:---|
| `name` | `str` | Yes | — | Class name (e.g., `"OrderService"`) or FQN. |

**Returns:** Class doc with methods, annotations, dependencies, `callsOut`/`calledBy` edges, and citation.

---

### 🔧 `get_file`

**Fetch all indexed chunks for a file path.**

| Parameter | Type | Required | Default | Description |
|:---|:---:|:---:|:---:|:---|
| `path` | `str` | Yes | — | File path as indexed (e.g., `"src/main/java/com/example/OrderController.java"`). |

**Returns:** List of all chunks (class, method, etc.) for that file.

---

### 🔧 `list_services`

**Inventory of all `@Service` beans in the indexed codebase.**

| Parameter | Type | Required | Default | Description |
|:---|:---:|:---:|:---:|:---|
| *(none)* | — | — | — | Takes no parameters. |

**Returns:** List of all service class docs.

---

### 🔧 `stats`

**Index statistics — doc counts, chunk types, modules.**

| Parameter | Type | Required | Default | Description |
|:---|:---:|:---:|:---:|:---|
| *(none)* | — | — | — | Takes no parameters. |

**Returns:** `product`, `raw_collection`, `serving_collection`, `total_docs`, `by_chunk_type`, `modules[]`.

---

## 🔀 Pluggable Providers

The platform **core** is fully runnable with just `git` + `docker compose`. Three concerns are behind clean provider interfaces — each with a built-in default and an optional MCP adapter.

```
┌─────────────────────────────────────────────────────────────┐
│                    ontology-rag-mcp CORE                    │
│         (always works: git + docker compose + CLI)          │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌─────────────┐  ┌──────────────┐  ┌───────────────────┐  │
│  │  SOURCE     │  │ ORCHESTRATOR │  │  DEPLOY           │  │
│  │  PROVIDER   │  │              │  │  PROVIDER         │  │
│  ├─────────────┤  ├──────────────┤  ├───────────────────┤  │
│  │ local (git) │  │ local (CLI)  │  │ docker (compose)  │  │
│  │ github (MCP)│  │ jenkins (MCP)│  │ ssh (MCP)         │  │
│  └─────────────┘  └──────────────┘  └───────────────────┘  │
│       ↓ optional       ↓ optional        ↓ optional        │
│   GitHub MCP       Jenkins MCP       Linux SSH MCP         │
└─────────────────────────────────────────────────────────────┘
```

| Concern | Env Var | Default | MCP Adapter | Degrades To |
|---|---|---|---|---|
| **Source** | `SOURCE_PROVIDER` | `local` | `github` → GitHub MCP | GitPython clone |
| **Orchestration** | `ORCHESTRATOR` | `local` | `jenkins` → Jenkins MCP | CLI commands |
| **Deploy** | `DEPLOY_PROVIDER` | `docker` | `ssh` → Linux SSH MCP | docker compose |

> MCP adapters are **first-class optional integrations**, not core dependencies. They are used (a) as agents during development/testing and (b) for headless "GitHub URL + target host → live MCP URL" workflows.

### Configuring repos (`repos.yml`)

```yaml
repos:
  - url: https://github.com/spring-projects/spring-petclinic
    branch: main
    name: petclinic

  - url: https://github.com/your-org/order-service
    branch: main
    name: orders
    subpath: order-service    # optional: index only this subdirectory

  - url: https://github.com/your-org/payment-service
    branch: develop
    name: payments
```

---

## 💬 Sample Queries

These queries are from the acceptance criteria. Use them in Cursor chat or via `ontology-rag ask`.

### Endpoint discovery

```
What REST endpoints does this expose?
```

**Expected:** `find_rest_endpoints` returns a list of routes with HTTP methods, controller classes, and flow names.

### Request flow (anchorless)

```
Walk me through what happens when a user creates an order.
```

**Expected:** `flow_of` or `rag_search` (with flow-doc seeding) returns the Controller→Service→Repository chain and a Mermaid sequence diagram — even though the query names no specific class.

### Cross-service relationships

```
Which service calls the payment service?
```

**Expected:** `uses_of` / `callers_of` returns Feign/RestTemplate edges between microservices.

### Class explanation

```
Explain the OrderService class.
```

**Expected:** `get_class` returns the class doc, methods, injected dependencies, and collaborator edges.

### Configuration

```
What database is configured in application.yml?
```

**Expected:** `rag_search` with `config` intent returns the relevant configuration chunk with file citation.

---

## 📁 Project Structure

```
ontology-rag-mcp/
├── ontology_core/              # Config, Solr client, embeddings, retrieval, intent, MCP server, CLI
│   ├── config.py               # Pydantic settings from env vars
│   ├── models.py               # Shared DTOs (SearchHit, FlowResult, etc.)
│   ├── embeddings.py           # Local sentence-transformers embedder
│   ├── corrections.py          # LLM-verify + runtime re-embed of user fixes
│   ├── intent.py               # Offline regex intent classifier
│   ├── retrieval.py            # Hybrid BM25 + kNN + RRF + flow-aware fusion
│   ├── llm.py                  # Optional OpenAI-compatible LLM client
│   ├── mcp_server.py           # FastMCP server + 13 tools
│   ├── cli.py                  # ontology-rag CLI (ingest / serve / ask)
│   └── solr/
│       ├── client.py           # Solr 9 REST client
│       └── managed-schema.xml  # Collection schema
├── ontology_ingest/            # Source providers, parser, flow ontology, pipeline
│   ├── source/
│   │   ├── base.py             # SourceProvider interface + factory
│   │   ├── local_git.py        # GitPython shallow clone (default)
│   │   └── github_mcp.py       # Optional GitHub MCP adapter
│   ├── parser/
│   │   ├── java_parser.py      # javalang Java/Spring parser
│   │   └── spring_annotations.py
│   ├── ontology/
│   │   └── flow_builder.py     # REST flow tracing + call graph
│   ├── chunker.py              # Parsed artifacts → Solr docs
│   ├── pipeline.py             # End-to-end ingest orchestrator
│   └── embed_pipeline.py       # Raw → serving collection with vectors
├── ontology_deploy/            # Deploy providers + Jenkins orchestrator
│   ├── base.py                 # DeployProvider interface
│   ├── docker_provider.py      # docker compose (default)
│   ├── ssh_provider.py         # Optional Linux SSH MCP adapter
│   └── jenkins_orchestrator.py # Optional Jenkins MCP adapter
├── docker-compose.yml          # Solr 9.6 + app
├── Dockerfile
├── repos.yml                   # Default repos to ingest
├── Jenkinsfile                 # CI/CD pipeline for headless deploy
├── .env.example                # All env vars documented
├── pyproject.toml
├── tests/
│   ├── test_java_parser.py
│   ├── test_spring_annotations.py
│   ├── test_flow_builder.py
│   └── test_intent.py
└── README.md
```

---

## 🛠 Tech Choices

| Choice | Rationale |
|---|---|
| **javalang** | Pure Python, zero native bindings. Sufficient for Spring annotation/method/dependency extraction. tree-sitter is more robust for partial parses but adds build complexity — reserved for future pluggable parser interface. |
| **Apache Solr 9** | BM25 + dense vector kNN in one stack. Graph stored as document fields (`callsOut`, `calledBy`, `flowName`) — no separate graph DB. |
| **sentence-transformers** | Offline embeddings with `BAAI/bge-small-en-v1.5` (384-dim). No paid API. Graceful zero-vector fallback if model unavailable. |
| **FastMCP** | Official MCP Python SDK. Supports streamable HTTP and stdio transports. |
| **GitPython** | Shallow clone for reproducible, MCP-agnostic source fetching. |
| **Pydantic** | Typed settings, DTOs, and tool return values. |

---

## 🧪 Development & Testing

```powershell
# Install with dev dependencies
pip install -e ".[dev]"

# Run unit tests
pytest tests/ -v

# Lint
ruff check .

# Ingest a repo
ontology-rag ingest --repos https://github.com/spring-projects/spring-petclinic

# Start MCP server
ontology-rag serve

# One-shot query
ontology-rag ask "What REST endpoints does this expose?"
```

**Test coverage:**

| Test File | Covers |
|---|---|
| `test_java_parser.py` | Controller/service parsing, HTTP mapping extraction, call detection |
| `test_spring_annotations.py` | `@RequestMapping`, `@GetMapping`, stereotype detection |
| `test_flow_builder.py` | Controller→Service flow tracing, call graph edges |
| `test_intent.py` | Query intent classification (flow/endpoint/class/config) |

---

## 🔧 Troubleshooting

### Solr not ready

**Symptom:** `TimeoutError: Solr not ready` during ingest.

**Fix:**
```powershell
# Check Solr health
curl http://localhost:8983/solr/admin/info/system

# Restart Solr container
docker compose restart solr

# Wait for healthy status
docker compose ps
```

### Embedding model download slow

**Symptom:** First ingest hangs on "Loading embedding model."

**Fix:** The first run downloads ~130 MB for `BAAI/bge-small-en-v1.5`. Subsequent runs use the cached model. Ensure internet access on first run, or pre-download:

```powershell
python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('BAAI/bge-small-en-v1.5')"
```

### MCP server not connecting in Cursor

**Symptom:** `ontology-rag` shows red/disconnected in Cursor MCP settings.

**Fixes:**
1. Verify the server is running: `curl http://localhost:8765/mcp`
2. Check `MCP_PORT` matches your `mcp.json` URL
3. For stdio mode, ensure `ontology-rag` is in your PATH
4. Restart Cursor after config changes

### Empty search results

**Symptom:** `rag_search` returns 0 hits.

**Fixes:**
1. Run ingest first: `ontology-rag ingest`
2. Check index stats: `ontology-rag ask` won't work, but MCP `stats` tool shows doc counts
3. Verify `RAG_PRODUCT` matches the ingested product name
4. Check Solr directly: `curl "http://localhost:8983/solr/petclinic-rag/select?q=*:*&rows=0"`

### No flow results for vague questions

**Symptom:** "walk me through..." returns class docs instead of flow docs.

**Fix:** Ensure `RAG_FLOW_SEED=true` (default). The retriever injects a flow/endpoint doc when intent is `flow` but no flow doc is in the top results.

### Private repo access denied

**Symptom:** Git clone fails with 401/403.

**Fix:** Set `GITHUB_TOKEN` in `.env`:
```
GITHUB_TOKEN=ghp_your_token_here
```

---

## 👥 Author & Contact

This project was created by **Pawan Gunjkar**.

- **Author**: Pawan Gunjkar
- **Email**: pawangunjkar@gamil.com
- **Bug Reports**: If you find any bugs, issues, or want to request help, please report them at the email above.

---

## 📄 License

MIT License — see [LICENSE](LICENSE) for details.

---

<div align="center">

**Built with FastMCP + Apache Solr 9 + sentence-transformers**

*Turn any Spring Boot repo into a queryable code ontology.*

</div>