🧠 Enterprise Knowledge Integrator

Connect Corporate Private Data (PDF, Excel, Word, SQL) to LLMs & AI Agents with Built-in PII Sanitization, Hybrid Search & MCP Server.

Live Dashboard • Quickstart • MCP Server Setup • Architecture • API Reference

🌟 Why Enterprise Knowledge Integrator?

Enterprises have fragmented knowledge across PDF policy documents, Excel/CSV financial models, contracts, and ERP notes. Feeding this data directly into LLMs often leads to PII leakage, hallucinations on numbers, and context loss.

Enterprise Knowledge Integrator is an open-source, lightweight, plug-and-play middleware that turns your company's raw files into clean, cited, and secure context for any LLM or AI Agent.

Related MCP server: doc-intel MCP server

✨ Key Features

• 📄 Tabular-Aware ETL (Excel & CSV): Converts spreadsheet rows into markdown tables and semantic row groups so LLMs never hallucinate row numbers or formulas.

• 🛡️ Built-in PII & Secret Sanitizer: Automatically detects and masks TCKN, IBANs, Credit Cards, Tax IDs (VKN), phone numbers, and API keys before embedding or prompt injection.

• ⚡ Hybrid Retrieval (Vector + Okapi BM25 + RRF): Blends dense embeddings with sparse keyword matching using Reciprocal Rank Fusion (RRF) for 100% accuracy on financial codes and numbers.

• 👥 Role-Based Access Control (RBAC): Enforces document clearance levels (Public, Internal, Confidential, Restricted) and department filtering.

• 🔍 Citation & Hallucination Validator: Automatically checks generated LLM answers against source documents and calculates a confidence score.

• 🔄 Directory Auto-Watcher: Monitors your folders/cloud drive mounts and automatically re-indexes added or modified files.

• 🔌 Universal Gateways:

  • Model Context Protocol (MCP) for Cursor, Claude Desktop, Antigravity.

  • FastAPI REST API with Swagger UI.

  • Interactive Web Dashboard (Zero extra dependencies required).

  • LangChain / LangGraph Tool Adapter.

🏛️ System Architecture

graph TD
    subgraph Ingestion ["1. Multi-Source Ingestion & ETL"]
        F1["📄 Documents (PDF, Word, Markdown)"]
        F2["📊 Tabular (Excel, CSV)"]
        F3["🗄️ Notes & Text Snippets"]
        F1 & F2 & F3 --> PII["🛡️ PII Masker (TCKN, IBAN, Cards)"]
        PII --> Chunk["✂️ Semantic & Parent-Child Chunker"]
    end

    subgraph Storage ["2. Storage & Hybrid Search Engine"]
        Chunk --> V["V-Store: Cosine Dense Embeddings"]
        Chunk --> B["BM25: Sparse Keyword Index"]
        V & B --> RRF["🎯 Reciprocal Rank Fusion (RRF)"]
    end

    subgraph Governance ["3. Security & Governance"]
        RRF --> RBAC["👥 RBAC & Clearance Filter"]
        RBAC --> Val["🔍 Citation & Grounding Validator"]
    end

    subgraph Interfaces ["4. LLM & Agent Gateways"]
        Val --> MCP["⚡ MCP Server (Claude Desktop / Cursor)"]
        Val --> API["🌐 FastAPI REST API (/api/v1/context)"]
        Val --> UI["🖥️ Modern Web Dashboard (/dashboard)"]
        Val --> SDK["💼 LangChain / LangGraph Adapter"]
    end

⚡ Quickstart in 60 Seconds

1. Installation

git clone https://github.com/your-username/enterprise-knowledge-integrator.git
cd enterprise-knowledge-integrator
pip install -r requirements.txt

2. Launch the Web Dashboard & API

python -m knowledge_integrator.interfaces.api.app

Open your browser at http://localhost:8088/dashboard to access the visual control panel.

💻 CLI Usage

Ingest a Text / Policy Note:

python -m knowledge_integrator.interfaces.cli.main ingest-text \
  --title "2025 Travel Policy" \
  --content "Daily travel allowance is 2,500 TL. Stays above 5,000 TL require CFO approval." \
  --category "policy"

Ingest Files or Directories (PDF, Excel, CSV, Word, Markdown):

python -m knowledge_integrator.interfaces.cli.main ingest ./company_docs/ --category "finance"

Search Knowledge Base:

python -m knowledge_integrator.interfaces.cli.main query "What is the travel budget limit?"

List Indexed Documents:

python -m knowledge_integrator.interfaces.cli.main list

⚡ Model Context Protocol (MCP) Server

Connect your corporate knowledge directly into Claude Desktop, Cursor IDE, or Antigravity.

Add this to your claude_desktop_config.json or cursor settings:

{
  "mcpServers": {
    "company-knowledge": {
      "command": "python",
      "args": ["-m", "knowledge_integrator.interfaces.cli.main", "serve-mcp"]
    }
  }
}

Available MCP Tools:

• search_company_knowledge: Performs hybrid search on private company documents.

• get_company_context: Returns clean, cited context ready for prompt injection.

• list_company_documents: Lists all indexed sources and metadata.

• ingest_company_note: Dynamically saves a new policy or knowledge snippet.

🌐 REST API Reference

Interactive Swagger documentation available at: http://localhost:8088/docs

🤖 Python & LangChain / LangGraph Integration

from knowledge_integrator import KnowledgeEngine
from knowledge_integrator.agentic_cfo_adapter import AgenticCFOKnowledgeAdapter

# 1. Initialize engine
engine = KnowledgeEngine()

# 2. Ingest document
engine.ingest_file("budget_2025.xlsx", category="finance")

# 3. Retrieve LLM context
ctx = engine.get_context_for_llm("What was the Q3 software budget?")
print(ctx.context_text)

# 4. Use as a LangChain / LangGraph Tool for AI Agents
adapter = AgenticCFOKnowledgeAdapter(engine)
agent_tool = adapter.as_langchain_tool()

🐳 Docker Deployment

docker-compose up -d

🧪 Running Tests

python -m pytest knowledge_integrator/tests/ -v

📄 License

This project is licensed under the MIT License — see the LICENSE file for details.