AI Due Diligence Copilot

AI Due Diligence Copilot is an end-to-end financial document analysis system that ingests corporate filings (10-K / 10-Q), extracts structured financial information, retrieves supporting evidence, and answers analyst questions using the most reliable source available.

Unlike many AI applications that rely entirely on an LLM, this system intelligently routes each question to either:

• Structured financial data stored in PostgreSQL

• Retrieved filing evidence through a Retrieval-Augmented Generation (RAG) pipeline

• A combination of both

To improve trustworthiness, generated answers can be evaluated using a PyTorch-based groundedness classifier that checks whether claims are supported by retrieved evidence.

Why This Project?

Financial analysts frequently need answers that are:

• Factually accurate

• Explainable

• Traceable to source documents

Traditional LLM-based assistants can hallucinate numbers or provide unsupported claims.

This project addresses that problem by:

• Using SQL for quantitative reasoning

• Using RAG for qualitative document understanding

• Using groundedness evaluation to assess evidence support

• Supporting optional Claude-based answer synthesis while remaining fully functional without any paid API

For deeper implementation details and design decisions, see:

ARCHITECTURE.md

Related MCP server: SEC-MCP

Features

Document Ingestion

• Upload financial filings (.txt, .pdf, .md)

• Automatic document cleaning and chunking

• Filing metadata tracking

• Persistent PostgreSQL storage

Structured Financial Metric Extraction

Currently extracts and stores:

• Revenue

• Operating Margin

• Net Income

• R&D Expense

• Cash & Cash Equivalents

These metrics are stored in PostgreSQL and can be queried directly for quantitative analysis.

Intelligent Query Routing

The system automatically determines whether a question should be answered through:

• Structured SQL retrieval

• Document retrieval (RAG)

• SQL + RAG

Examples:

MCP Tool Integration

Implements Model Context Protocol (MCP) tools:

• Company Lookup

• Financial Ratio Calculator

• Document Search

Groundedness Evaluation

A lightweight PyTorch classifier evaluates whether generated claims are supported by retrieved evidence.

Outputs include:

• Groundedness scores

• Claim-level support classification

• Confidence indicators

Evaluation Harness

Built-in evaluation framework measuring:

• Routing accuracy

• Retrieval relevance

• Groundedness

• Latency

Interactive Dashboard

Web interface for:

• Filing uploads

• Question answering

• Viewing routing decisions

• Viewing tool usage

• Viewing groundedness scores

System Architecture

                    Financial Filing
                           │
                           ▼
                  Document Ingestion
                           │
                           ▼
                      Chunking
                           │
                           ▼
       Metric Extraction + Vectorization
                 (TF-IDF + SVD)
                 │              │
                 ▼              ▼
          PostgreSQL      Vector Store
                 ▲              ▲
                 │              │
User Query ───► Query Router ───┘
                 │
      ┌──────────┴──────────┐
      ▼                     ▼

 SQL Financial Route     RAG Retrieval

      ▼                     ▼

 Financial Metrics     Evidence Search

      └──────────┬──────────┘
                 ▼

         Answer Generation

                 ▼

      Groundedness Evaluation

                 ▼

          Final Response

Answer Generation Modes

1. Offline / Local Mode (Default)

No API keys required.

The system answers questions using:

• PostgreSQL financial data

• RAG retrieval

• Extractive answer generation

• PyTorch groundedness evaluation

This mode was used during development and testing.

2. Claude-Assisted Mode (Optional)

If an Anthropic API key is provided:

ANTHROPIC_API_KEY=your_api_key_here

retrieved evidence can be passed to Claude for natural-language answer generation.

Pipeline:

User Query
    ↓
Retrieval / SQL
    ↓
Claude
    ↓
Groundedness Evaluation
    ↓
Final Response

Claude is only used for answer synthesis.

The system does not depend on Claude for:

• Retrieval

• Query routing

• Financial calculations

• Groundedness scoring

If no API key is present, the application automatically falls back to the local extractive pipeline.

Tech Stack

Backend

Database

Machine Learning

Retrieval

AI Tooling

Project Structure

diligence-copilot/
│
├── app/
│   ├── db/
│   ├── ingestion/
│   ├── ml/
│   ├── rag/
│   ├── mcp_tools/
│   └── main.py
│
├── data/
│
├── scripts/
│   └── setup_postgres.sql
│
├── tests/
│
├── Dockerfile
├── docker-compose.yml
├── requirements.txt
├── ARCHITECTURE.md
└── README.md

Installation

Prerequisites

• Python 3.11+

• PostgreSQL 16+ (tested on PostgreSQL 17.11)

• Git

1. Clone Repository

git clone <repository-url>
cd diligence-copilot

2. Create Virtual Environment

Windows

python -m venv venv
.\venv\Scripts\Activate.ps1

Linux / macOS

python -m venv venv
source venv/bin/activate

3. Install Dependencies

Install CPU-only PyTorch:

pip install torch --index-url https://download.pytorch.org/whl/cpu

Install project requirements:

pip install -r requirements.txt

4. Configure PostgreSQL

Run:

psql -U postgres -f scripts/setup_postgres.sql

This creates:

• diligence_copilot database

• diligence_app user

• Required permissions

5. Build Groundedness Dataset

python -m app.ml.build_dataset

6. Train Groundedness Classifier

python -m app.ml.groundedness

7. Start Application

uvicorn app.main:app --reload

Application:

http://localhost:8000

API Documentation:

http://localhost:8000/docs

Docker

docker-compose up --build

Verification Walkthrough

Create Test Filing

Total revenue for fiscal year 2024 was $500 million.

Operating margin for fiscal year 2024 was 12.5%, compared to 10.1% in fiscal year 2023.

The company faces significant risk from a single supplier located in Vietnam.

Upload Filing

Use:

• Ticker: TEST

• Company: Test Company

• Fiscal Period: FY2024

Example output:

Done: 1 chunks, 3 financial metrics extracted.

Test Financial Reasoning

Question:

What was the change in operating margin?

Example output:

operating_margin moved from 10.1 (FY2023)
to 12.5 (FY2024), a change of 23.76%.

Route:

SQL

Tool:

financial_ratio_calculator

Test Retrieval

Question:

What supplier risks does the company face?

Example output:

The company faces significant risk from a single supplier located in Vietnam.

Note: For very small filings that fit into a single chunk, retrieval may return the entire chunk rather than a single sentence.

Route:

RAG

Run Evaluation

python -m app.eval.run_eval

Results

Validated end-to-end on:

• Document ingestion

• Financial metric extraction

• PostgreSQL persistence

• Query routing

• SQL-based financial reasoning

• Retrieval-based risk analysis

• Groundedness evaluation

• FastAPI deployment

Sample Evaluation Results

Groundedness Classifier

Results vary slightly because the evaluation dataset is intentionally small.

Limitations

• Groundedness classifier trained on 38 labeled examples

• Financial metric extraction currently uses rule-based patterns

• Retrieval uses TF-IDF + SVD instead of transformer embeddings

• Vector index is in-memory and optimized for demonstration-scale workloads

• Claude integration is optional and not required for core functionality

Future Improvements

• Transformer-based embeddings

• Hybrid search (keyword + vector)

• Larger groundedness datasets

• LLM-based extraction fallback

• Multi-step agent workflows

• pgvector integration

• Multi-document comparison

• Automated analyst report generation