Question
   ↓
Vector Search
   ↓
LLM
   ↓
Answer

Question
   ↓
LangGraph Agent
   ↓
Tool Selection
   ├── Document Retrieval
   ├── Calculator
   ├── Web Search
   └── OCR
   ↓
Grounded Response + Citations + Trace

flowchart TB
    U[User]

    U --> UI[Streamlit UI]
    U --> API[FastAPI API]
    UI --> API

    API --> ING[Ingestion]
    API --> RAG[RAG Query Pipeline]
    API --> AGENT[LangGraph Agent]

    ING --> LOAD[Load PDF / Image / TXT / MD]
    LOAD --> OCR[Tesseract OCR when needed]
    OCR --> CHUNK[Recursive Chunking]
    CHUNK --> EMB[Ollama Embeddings]
    EMB --> Q[(Qdrant)]

    RAG --> RET[Vector Retrieval]
    RET --> Q
    RET --> RR[Cross-Encoder Reranker]
    RR --> GEN[Ollama Generation]
    GEN --> RESP[Grounded Answer + Citations]

    AGENT --> TOOLS[Shared Tool Layer]
    TOOLS --> RET
    TOOLS --> CALC[Calculator]
    TOOLS --> WEB[Tavily Search]
    TOOLS --> AOCR[On-demand OCR]

    MCP[MCP Client] --> SERVER[FastMCP Server]
    SERVER --> TOOLS

    RESP --> TRACE[JSONL Run Trace]
    AGENT --> TRACE

flowchart LR
    D[Uploaded File] --> TYPE{Format}

    TYPE -->|PDF| PDF[pypdf]
    TYPE -->|Image| IMG[Tesseract OCR]
    TYPE -->|TXT / MD| TXT[Text Loader]

    PDF --> CHECK{Extracted text < 20 chars?}
    CHECK -->|Yes| POCR[OCR PDF Page]
    CHECK -->|No| PLAIN[Extracted Page Text]

    POCR --> CHUNK[800-char chunks<br/>120-char overlap]
    PLAIN --> CHUNK
    IMG --> CHUNK
    TXT --> CHUNK

    CHUNK --> EMB[Ollama Embedding]
    EMB --> STORE[(Qdrant)]

    Q[User Query] --> QEMB[Query Embedding]
    QEMB --> SEARCH[Top-K Vector Search]
    STORE --> SEARCH
    SEARCH --> CAND[Up to 20 candidates]
    CAND --> RANK[Cross-Encoder Reranking]
    RANK --> FINAL[Top 5 chunks]
    FINAL --> CONTEXT[Grounding Context]
    CONTEXT --> LLM[Ollama LLM]
    LLM --> ANSWER[Answer + Citation Metadata]

flowchart TD
    START[User Message] --> AGENT[LangGraph ReAct Agent]
    AGENT --> DECIDE{Need a tool?}

    DECIDE -->|Document question| RET[retrieve_documents]
    DECIDE -->|Arithmetic| CALC[calculator]
    DECIDE -->|External information| WEB[web_search]
    DECIDE -->|Scanned file| OCR[ocr_scan]
    DECIDE -->|No| FINAL[Final Answer]

    RET --> AGENT
    CALC --> AGENT
    WEB --> AGENT
    OCR --> AGENT

    AGENT --> CHECK{Enough evidence?}
    CHECK -->|No| DECIDE
    CHECK -->|Yes| FINAL

    RET --> CITE[Chunk IDs + metadata]
    CITE --> FINAL
    FINAL --> TRACE[Run ID + tool-call trace]

sequenceDiagram
    participant C as MCP Client
    participant S as DocAgent FastMCP
    participant T as Shared Tools
    participant Q as Qdrant
    participant W as Tavily
    participant F as Server Files

    C->>S: Call tool

    alt retrieve_documents
        S->>T: retrieve_documents(query, doc_id?)
        T->>Q: Vector search + filtering
        Q-->>T: Candidate chunks
        T-->>S: Ranked document evidence
    else calculator
        S->>T: calculator(expression)
        T-->>S: Safe arithmetic result
    else web_search
        S->>T: web_search(query)
        T->>W: Tavily search
        W-->>T: Search results
        T-->>S: Web evidence
    else ocr_scan
        S->>T: ocr_scan(file_path, page)
        T->>F: Read file
        T-->>T: Tesseract OCR
        T-->>S: Extracted text
    end

    S-->>C: Structured tool result

Query
  ↓
Ollama embedding
  ↓
Qdrant similarity search
  ↓
Up to 20 candidates
  ↓
Cross-encoder reranking
  ↓
Top 5 passages

+  -  *  /  **  %
()

http://localhost:8000/docs

curl -X POST "http://localhost:8000/ingest" \
  -F "file=@document.pdf"

curl -X POST "http://localhost:8000/query" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What does the document say about deployment?"
  }'

curl -X POST "http://localhost:8000/query" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Summarize the deployment process.",
    "doc_id": "DOCUMENT_ID"
  }'

curl -X POST "http://localhost:8000/agent/chat" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Find the relevant information and calculate the total."
  }'

git clone https://github.com/hardikkaurani/DocAgent.git
cd DocAgent

cp .env.example .env

OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_LLM_MODEL=llama3.1
OLLAMA_EMBED_MODEL=nomic-embed-text

QDRANT_URL=http://localhost:6333
QDRANT_COLLECTION=docagent_chunks

RERANKER_MODEL=cross-encoder/ms-marco-MiniLM-L-6-v2

TOP_K_RETRIEVE=20
TOP_K_RERANK=5

TAVILY_API_KEY=
LANGCHAIN_API_KEY=
LANGCHAIN_PROJECT=docagent

UPLOAD_DIR=./data/uploads
RUNS_DIR=./runs

docker compose up -d

./scripts/pull_ollama_models.sh

python3 -m venv .venv
source .venv/bin/activate

pip install -r requirements-dev.txt

docker compose up -d qdrant ollama

uvicorn app.main:app --reload

streamlit run streamlit_app/app.py

python -m app.mcp_server

pytest

runs/
└── YYYY-MM-DD.jsonl

LANGCHAIN_API_KEY=...
LANGCHAIN_PROJECT=docagent

DocAgent/
├── app/
│   ├── agent/
│   │   ├── graph.py
│   │   └── tools.py
│   ├── api/
│   │   ├── routes_agent.py
│   │   ├── routes_health.py
│   │   ├── routes_ingest.py
│   │   └── routes_query.py
│   ├── generation/
│   │   ├── llm.py
│   │   └── qa_chain.py
│   ├── ingestion/
│   │   ├── chunking.py
│   │   ├── loaders.py
│   │   └── ocr.py
│   ├── retrieval/
│   │   ├── reranker.py
│   │   └── retriever.py
│   ├── tracing/
│   │   └── tracer.py
│   ├── vectorstore/
│   │   ├── embeddings.py
│   │   └── qdrant_store.py
│   ├── config.py
│   ├── main.py
│   ├── mcp_server.py
│   └── schemas.py
├── streamlit_app/
│   └── app.py
├── tests/
├── scripts/
│   └── pull_ollama_models.sh
├── data/
├── Dockerfile
├── Dockerfile.streamlit
├── docker-compose.yml
├── requirements.txt
├── requirements-dev.txt
├── .env.example
├── .gitignore
├── LICENSE
└── README.md

chunk_id
doc_id
filename
page
text
score