Agentic Financial Research Assistant

An agentic system built on LangChain + LangGraph that plans, retrieves, calculates, and compares across RBI financial documents and live market data — with MCP server, guardrails, multi-turn memory, Yahoo Finance integration, portfolio analysis, and automated evaluation.

What It Does

Financial analysts spend hours manually cross-referencing RBI annual reports, monetary policy statements, and live market data. This agent replaces brittle prompt-engineering with an explicit state machine that deterministically routes queries to the right tool, validates outputs, and escalates low-confidence answers to human review.

Architecture

graph LR
    Start([User Query]) --> Sanitize[sanitize_state]
    Sanitize --> Memory[memory_resolver]
    Memory --> Guard[guardrail_check]
    Guard -->|continue| Planner[planner]

    Planner -->|rag_search| RAG[rag_search<br/>OpenSearch Hybrid<br/>BM25 + HNSW kNN]
    Planner -->|financial_calculator| Calc[financial_calculator<br/>Safe AST Eval]
    Planner -->|document_comparator| Comp[document_comparator<br/>Gemini-based]
    Planner -->|web_search| Web[web_search<br/>DuckDuckGo]
    Planner -->|yahoo_finance| YF[yahoo_finance<br/>Live Stock Data]
    Planner -->|portfolio_analyzer| PA[portfolio_analyzer<br/>Sharpe & Risk]
    Planner -->|final_answer| Final[final_answer]

    RAG --> Guard
    Calc --> Guard
    Comp --> Guard
    Web --> Guard
    YF --> Guard
    PA --> Guard

    Guard -->|continue| Planner
    Guard -->|respond| Final
    Guard -->|human_review| Human[human_review<br/>Enterprise HITL]

    Human --> End([END])
    Final --> End

11-node LangGraph with conditional edges. Every tool execution routes through a guardrail checkpoint before returning to the planner.

Key Features

1. Async-First Architecture

The entire agent graph is built for async execution:

• Async LLM calls — call_llm_async() with asyncio.to_thread() offloads blocking Google client to worker threads, using non-blocking asyncio.sleep for rate-limit backoff

• Async retrieval — retrieve_passages_async() runs OpenSearch queries without blocking the event loop

• Async portfolio analysis — PortfolioAnalyzerTool._arun() runs heavy CPU + I/O (yfinance + numpy) in a worker thread via asyncio.to_thread()

• Parallel retrieval — parallel_retrieve() uses asyncio.gather() to fetch multiple queries concurrently

• FastAPI async endpoints — api/main.py serves requests with full async/await support

2. Fast-Path Planner (~70% LLM Call Reduction)

The planner uses 9 deterministic fast-paths that skip the LLM entirely for common query patterns:

Tradeoff: Fast-paths add maintenance overhead but save ~1-3s per query by avoiding LLM planner calls for obvious patterns.

3. Guardrails — Hard Caps, Not Soft Hints

Soft hints are ignored by LLMs. Hard caps are deterministic, auditable, and prevent runaway costs.

4. Multi-Turn Memory

• LLM-based coreference resolution — resolves pronouns ("those two", "that", "previous year")

• Regex fallback — for simple patterns when LLM is unavailable

• Sliding window — keeps last 5 turns, summarizes older turns

• Accumulated state — retrieved_passages, calculation_results, retrieved_contexts persist across turns

Example multi-turn trace:

Turn 1: "What was the repo rate in FY2023?" → 6.5%
Turn 2: "And what about the previous year?" → 4.0% (resolved FY2022)
Turn 3: "What's the percentage increase between those two?" → 62.5%

5. RAG Pipeline — OpenSearch Hybrid Search

Built on OpenSearch 2.x with a multi-stage pipeline:

Query → Router → [HyDE] → OpenSearch (BM25 + kNN hybrid) → Rerank → CRAG → Cache → Return

Query-specific strategies:

6. Yahoo Finance + Portfolio Analyzer

Supported tickers: .NS (NSE India), .BO (BSE India), US tickers, indices (^NSEI).

7. MCP Server

All tools exposed as an MCP server for universal agent compatibility:

# Any MCP client can call:
await search_financial_documents("RBI repo rate", top_k=5)
await calculate_financial_metric("growth_rate(4.0, 6.5)")
await compare_documents(doc_a="...", doc_b="...", metric="repo")
await yahoo_finance(ticker="RELIANCE.NS", operation="quote")
await portfolio_analyzer(tickers="RELIANCE.NS,INFY.NS")

Quick Start

Docker Compose (Recommended)

# 1. Clone
git clone https://github.com/Ajay-Kumar64/Agentic-Financial-Research-Assistant.git
cd Agentic-Financial-Research-Assistant

# 2. Environment
cp .env.example .env
# Add GOOGLE_API_KEY to .env
# Optional: Add LANGSMITH_API_KEY for tracing

# 3. Start all services
docker compose up --build

Services started:

• OpenSearch (:9200) — Vector + text search backend

• Redis (:6379) — Cache + conversation store with LRU eviction

• Agent (:8000) — FastAPI async backend

• UI (:8501) — Streamlit frontend

• MCP — MCP server for tool interoperability

Local Development

pip install -r requirements.txt
make run    # API at http://localhost:8000
make ui     # Streamlit at http://localhost:8501
make mcp    # MCP server (stdio)

# Run evaluations
make eval        # 18-metric evaluation
make eval-ragas  # RAGAS evaluation

Docker Architecture

Why multi-stage:

• UI and MCP don't download 2.2GB of embedding models

• Models pre-downloaded at build time — zero cold start

• Single Dockerfile, three targets via docker-compose.yml args

API Endpoints

Tech Stack

Project Structure

agentic-financial-assistant/
├── agent/
│   ├── graph.py                 # 11-node LangGraph state machine (async-ready)
│   ├── state.py                 # AgentState TypedDict
│   ├── planner_node.py          # Planner with 9 fast-paths + LLM fallback
│   ├── guardrails.py            # Loop, depth, token, latency, confidence checks
│   ├── llm_provider.py          # Gemini client with async + sync wrappers
│   ├── prompts/
│   │   ├── planner_system.txt
│   │   └── response_system.txt
│   └── tools/
│       ├── base.py              # BaseTool + ToolResult
│       ├── rag_search.py        # OpenSearch hybrid retrieval (async)
│       ├── calculator.py        # AST-based safe math evaluator
│       ├── comparator.py        # Gemini-based comparison
│       ├── web_search.py        # DuckDuckGo fallback
│       ├── memory.py            # Coreference resolution
│       ├── yahoo_finance.py     # Live stock data
│       └── portfolio_analyzer.py # Sharpe, volatility, drawdown (async _arun)
├── api/
│   ├── main.py                  # FastAPI with async endpoints
│   ├── models.py                # Pydantic schemas
│   └── middleware.py            # Request logging + error handling
├── ui/
│   └── app.py                   # Streamlit chat + trace viewer
├── mcp_server/
│   ├── server.py                # FastMCP with 5 tools
│   └── run.py                   # Entry point
├── rag/
│   ├── retriever.py             # SmartRetriever: Router → HyDE → OpenSearch → Rerank → CRAG → Cache
│   ├── opensearch_client.py     # OpenSearch connection + hybrid search + artifact export/import
│   ├── document_processor.py    # PDF → parent-child chunks + structured extraction
│   ├── indexing_pipeline.py     # Full indexing: process → embed → bulk index → artifact export
│   ├── reranker.py              # BGE cross-encoder with fast-path skip
│   ├── cache.py                 # Redis + in-memory LRU with graceful degradation
│   ├── config.py                # Centralized RAG config (env-driven)
│   └── fusion.py                # Reciprocal Rank Fusion (client-side fallback)
├── evaluation/
│   ├── golden_traces.json       # 20 test cases
│   ├── adversarial_inputs.json  # 10 safety tests
│   ├── metrics.py               # 18 metric functions
│   ├── judge.py                 # LLM-as-judge
│   └── run_eval.py              # Evaluation runner
├── eval/
│   └── ragas_eval.py            # RAGAS evaluation
├── tests/
│   ├── test_tools.py            # Unit + integration tests
│   ├── test_guardrails.py
│   ├── test_memory.py
│   ├── test_state.py
│   ├── test_mcp_server.py
│   ├── test_comparator.py
│   ├── test_adversarial.py
│   └── test_single_trace.py
├── docker-compose.yml           # OpenSearch + Redis + Agent + UI + MCP
├── Dockerfile                   # Multi-stage: builder → model-downloader → production
├── Makefile
├── requirements.txt
└── .env.example

Evaluation

18-Metric Framework (n=20 golden traces + 10 adversarial)

Pass rate: 12/19 (63%)

RAGAS Evaluation (n=20)

Adversarial Testing — 10/10 Passed

Known Limitations

1. Tool call redundancy is 10x over target. 51% vs. 5% target. The agent often retries the same tool with identical input (e.g., rag_search → empty → web_search with same query). The planner should reformulate queries before retrying.

2. Citation traceability gaps for non-RAG sources. 60% vs. 90% target. Citations are correctly shown when RAG documents are retrieved ([1], [Source: format). For web search results and LLM-generated responses, citations are inconsistent. The response assembler prompt needs structured output forcing for all source types.

3. Faithfulness scoring is misleading. 60% vs. 88% target. The LLM-as-judge conflates "correct refusal" ("I don't have enough information") with "unfaithful answer." The agent does not hallucinate — it either answers from retrieved documents or refuses. The judge needs calibration for correct refusal vs. hallucination.

4. Latency spikes under rate limits. p50 is ~2.5s, p95 is ~5s, but p99 spikes to 60s+ when Gemini API throttles. Exponential backoff with jitter is implemented but does not prevent the spike — it only makes it recoverable.

5. Intermediate step accuracy is misleadingly low. 24% vs. 90% target. The judge marks steps as "incorrect" when tools return empty results for off-topic queries. The tool executed correctly — there was no relevant data in the corpus. The judge conflates "empty result" with "wrong execution."

6. In-memory state. Conversation state is ephemeral and lost on server restart. Production deployments should migrate to Redis persistence.

Roadmap

Critical (Must Do)

• Fix tool call redundancy — add query reformulation before retry (target: <5%)

• Fix citation traceability for web/LLM sources — force structured output (target: >90%)

• Calibrate LLM-as-judge — distinguish correct refusal from hallucination (target: >85% faithfulness)

• Add circuit breaker for rate-limited LLM calls — cut p99 latency spikes

High Impact

• Add local lightweight classifier (DistilBERT) for routing — save 1-2 LLM calls per query

• Add streaming responses for long-running tool chains

• Migrate parent storage from in-memory to Redis/OpenSearch for persistence

Nice to Have

• A/B test Gemini vs. Claude 3.5 Sonnet for planner node

• Add Grafana dashboard for latency, cost, and guardrail metrics

• Add user satisfaction NPS to evaluation framework

• Integrate with QuantLib for advanced financial modeling

Future

• A2A multi-agent architecture for query decomposition

• Fine-tuned embedding model on RBI financial documents

• Real-time document ingestion pipeline (webhook-based)

• Upgrade pdfplumber → marker at 50+ docs, → docling at 500+ docs

License

MIT License — see LICENSE for details.