<div align="center">

# 🧠 Enterprise Knowledge Integrator

### *Connect Corporate Private Data (PDF, Excel, Word, SQL) to LLMs & AI Agents with Built-in PII Sanitization, Hybrid Search & MCP Server.*

[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
[![Python 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/)
[![FastAPI](https://img.shields.io/badge/FastAPI-0.111.0-009688.svg?logo=fastapi)](https://fastapi.tiangolo.com)
[![Model Context Protocol](https://img.shields.io/badge/MCP-Compatible-8A2BE2.svg)](https://modelcontextprotocol.io)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](http://makeapullrequest.com)

[**Live Dashboard**](#-interactive-web-dashboard) • [**Quickstart**](#-quickstart-in-60-seconds) • [**MCP Server Setup**](#-model-context-protocol-mcp-server) • [**Architecture**](#-system-architecture) • [**API Reference**](#-rest-api-reference)

---

</div>

## 🌟 Why Enterprise Knowledge Integrator?

Enterprises have fragmented knowledge across PDF policy documents, Excel/CSV financial models, contracts, and ERP notes. Feeding this data directly into LLMs often leads to **PII leakage**, **hallucinations on numbers**, and **context loss**.

**Enterprise Knowledge Integrator** is an open-source, lightweight, plug-and-play middleware that turns your company's raw files into clean, cited, and secure context for any LLM or AI Agent.

---

## ✨ Key Features

- 📄 **Tabular-Aware ETL (Excel & CSV)**: Converts spreadsheet rows into markdown tables and semantic row groups so LLMs never hallucinate row numbers or formulas.
- 🛡️ **Built-in PII & Secret Sanitizer**: Automatically detects and masks TCKN, IBANs, Credit Cards, Tax IDs (VKN), phone numbers, and API keys before embedding or prompt injection.
- ⚡ **Hybrid Retrieval (Vector + Okapi BM25 + RRF)**: Blends dense embeddings with sparse keyword matching using Reciprocal Rank Fusion (RRF) for 100% accuracy on financial codes and numbers.
- 👥 **Role-Based Access Control (RBAC)**: Enforces document clearance levels (`Public`, `Internal`, `Confidential`, `Restricted`) and department filtering.
- 🔍 **Citation & Hallucination Validator**: Automatically checks generated LLM answers against source documents and calculates a confidence score.
- 🔄 **Directory Auto-Watcher**: Monitors your folders/cloud drive mounts and automatically re-indexes added or modified files.
- 🔌 **Universal Gateways**:
  - **Model Context Protocol (MCP)** for Cursor, Claude Desktop, Antigravity.
  - **FastAPI REST API** with Swagger UI.
  - **Interactive Web Dashboard** (Zero extra dependencies required).
  - **LangChain / LangGraph Tool Adapter**.

---

## 🏛️ System Architecture

```mermaid
graph TD
    subgraph Ingestion ["1. Multi-Source Ingestion & ETL"]
        F1["📄 Documents (PDF, Word, Markdown)"]
        F2["📊 Tabular (Excel, CSV)"]
        F3["🗄️ Notes & Text Snippets"]
        F1 & F2 & F3 --> PII["🛡️ PII Masker (TCKN, IBAN, Cards)"]
        PII --> Chunk["✂️ Semantic & Parent-Child Chunker"]
    end

    subgraph Storage ["2. Storage & Hybrid Search Engine"]
        Chunk --> V["V-Store: Cosine Dense Embeddings"]
        Chunk --> B["BM25: Sparse Keyword Index"]
        V & B --> RRF["🎯 Reciprocal Rank Fusion (RRF)"]
    end

    subgraph Governance ["3. Security & Governance"]
        RRF --> RBAC["👥 RBAC & Clearance Filter"]
        RBAC --> Val["🔍 Citation & Grounding Validator"]
    end

    subgraph Interfaces ["4. LLM & Agent Gateways"]
        Val --> MCP["⚡ MCP Server (Claude Desktop / Cursor)"]
        Val --> API["🌐 FastAPI REST API (/api/v1/context)"]
        Val --> UI["🖥️ Modern Web Dashboard (/dashboard)"]
        Val --> SDK["💼 LangChain / LangGraph Adapter"]
    end
```

---

## ⚡ Quickstart in 60 Seconds

### 1. Installation

```bash
git clone https://github.com/your-username/enterprise-knowledge-integrator.git
cd enterprise-knowledge-integrator
pip install -r requirements.txt
```

### 2. Launch the Web Dashboard & API

```bash
python -m knowledge_integrator.interfaces.api.app
```

Open your browser at **`http://localhost:8088/dashboard`** to access the visual control panel.

---

## 💻 CLI Usage

#### Ingest a Text / Policy Note:
```bash
python -m knowledge_integrator.interfaces.cli.main ingest-text \
  --title "2025 Travel Policy" \
  --content "Daily travel allowance is 2,500 TL. Stays above 5,000 TL require CFO approval." \
  --category "policy"
```

#### Ingest Files or Directories (PDF, Excel, CSV, Word, Markdown):
```bash
python -m knowledge_integrator.interfaces.cli.main ingest ./company_docs/ --category "finance"
```

#### Search Knowledge Base:
```bash
python -m knowledge_integrator.interfaces.cli.main query "What is the travel budget limit?"
```

#### List Indexed Documents:
```bash
python -m knowledge_integrator.interfaces.cli.main list
```

---

## ⚡ Model Context Protocol (MCP) Server

Connect your corporate knowledge directly into **Claude Desktop**, **Cursor IDE**, or **Antigravity**.

Add this to your `claude_desktop_config.json` or `cursor settings`:

```json
{
  "mcpServers": {
    "company-knowledge": {
      "command": "python",
      "args": ["-m", "knowledge_integrator.interfaces.cli.main", "serve-mcp"]
    }
  }
}
```

### Available MCP Tools:
- `search_company_knowledge`: Performs hybrid search on private company documents.
- `get_company_context`: Returns clean, cited context ready for prompt injection.
- `list_company_documents`: Lists all indexed sources and metadata.
- `ingest_company_note`: Dynamically saves a new policy or knowledge snippet.

---

## 🌐 REST API Reference

| Method | Endpoint | Description |
| :--- | :--- | :--- |
| `POST` | `/api/v1/ingest/file` | Upload & index file (PDF, Excel, CSV, Word, MD) |
| `POST` | `/api/v1/ingest/text` | Ingest raw corporate note or rule |
| `POST` | `/api/v1/context` | Get cited LLM-ready context block |
| `POST` | `/api/v1/search` | Search ranked chunks (Hybrid) |
| `GET` | `/api/v1/documents` | List all indexed documents |
| `DELETE`| `/api/v1/documents/{id}`| Delete document and all associated embeddings |

Interactive Swagger documentation available at: `http://localhost:8088/docs`

---

## 🤖 Python & LangChain / LangGraph Integration

```python
from knowledge_integrator import KnowledgeEngine
from knowledge_integrator.agentic_cfo_adapter import AgenticCFOKnowledgeAdapter

# 1. Initialize engine
engine = KnowledgeEngine()

# 2. Ingest document
engine.ingest_file("budget_2025.xlsx", category="finance")

# 3. Retrieve LLM context
ctx = engine.get_context_for_llm("What was the Q3 software budget?")
print(ctx.context_text)

# 4. Use as a LangChain / LangGraph Tool for AI Agents
adapter = AgenticCFOKnowledgeAdapter(engine)
agent_tool = adapter.as_langchain_tool()
```

---

## 🐳 Docker Deployment

```bash
docker-compose up -d
```

---

## 🧪 Running Tests

```bash
python -m pytest knowledge_integrator/tests/ -v
```

---

## 📄 License

This project is licensed under the MIT License — see the [LICENSE](LICENSE) file for details.

<!-- verified-author: Enesp4rl4k -->