MCP-Powered Agentic RAG

An agentic Retrieval-Augmented Generation (RAG) system that combines a small curated machine learning knowledge base with real-time web search capabilities, powered by the Model Context Protocol (MCP).

Limitations of Naive RAG

Traditional RAG systems have several limitations:

1. Static Knowledge Base: Naive RAG relies solely on pre-indexed documents, making it unable to answer questions about recent events, current information, or topics not in the knowledge base.

2. No Tool Selection: These systems cannot intelligently decide when to use different information sources. They always query the same vector database regardless of the question type.

3. Limited Context Awareness: They lack the ability to understand query intent and route to appropriate tools (e.g., domain-specific knowledge base vs. general web search).

4. Single Source of Truth: All queries go through the same retrieval mechanism, even when the question might be better answered by external sources.

5. No Fallback Mechanism: If the knowledge base doesn't contain relevant information, the system fails rather than seeking alternative sources.

Related MCP server: Speakeasy Docs MCP

How Agentic RAG solves the Problem

Agentic RAG introduces intelligent decision-making and tool orchestration:

1. Multi-Source Intelligence: The system can choose between a curated knowledge base (for domain-specific questions) and web search (for general or current information).

2. Context-Aware Routing: An intelligent prompt guides the LLM to analyze query intent and route to the appropriate tool based on the question type.

3. Dynamic Information Retrieval: The system can fetch real-time information from the web when the knowledge base is insufficient.

4. Tool Orchestration: Through MCP, the system can seamlessly switch between different tools based on the query context.

5. Graceful Degradation: If one source fails, the system can automatically try alternative sources.

Solution Overview

This project implements an Agentic RAG system that:

• Maintains a small curated ML knowledge base (50 expert FAQs) in ChromaDB Cloud

• Provides real-time web search via Firecrawl for general queries

• Leverages MCP (Model Context Protocol) for seamless tool integration with Claude

The system acts as an intelligent assistant that knows when to use its specialized knowledge base versus when to search the web for general information not relevant to the knowledge base.

Workflow

1. User Query: User asks a question through Claude Desktop

2. Intent Analysis: Intelligent prompt analyzes the query to determine:

   • Is this an ML-related question? → Use ml_faq_retrieval

   • Is this a general question? → Use firecrawl_web_search

3. Tool Execution:

   • ML FAQ Tool: Queries ChromaDB Cloud, retrieves top 3 relevant FAQs

   • Web Search Tool: Searches the web via Firecrawl API

4. Return to User: Formatted response is returned through Claude

Tech Stack

• FastMCP: Fast Model Context Protocol framework for building MCP servers

• ChromaDB Cloud: Cloud-hosted vector database for storing and querying FAQ embeddings

• Firecrawl: Web scraping and search API for real-time information retrieval

Setup

Prerequisites

• Python 3.12 or higher

• uv package manager installed

• ChromaDB Cloud account (for API key, tenant, and database)

• Firecrawl API key

Installation

1. Clone and cd into the repository:

cd mcp-powered-agentic-rag

2. Install dependencies with uv:

uv pip install -r requirements.txt

Or use uv's project management:

uv sync

3. Set up environment variables: Create a .env file in the project root:

CHROMA_API_KEY=your_chroma_api_key
CHROMA_TENANT=your_chroma_tenant
CHROMA_DATABASE=your_chroma_database
FIRECRAWL_API_KEY=your_firecrawl_api_key

4. Verify setup:

uv run fastmcp dev server.py

Usage

Running the MCP Server

Development Mode (with Inspector)

uv run fastmcp dev server.py

Production Mode

uv run python server.py

Integrating with Claude Desktop

Add the following to your Claude Desktop MCP configuration:

{
  "mcpServers": {
    "mcp-rag": {
      "command": "/path/to/uv",
      "args": [
        "--directory",
        "/path/to/mcp-powered-agentic-rag",
        "run",
        "server.py"
      ]
    }
  }
}

Configuration

ChromaDB Cloud Setup

1. Create a ChromaDB Cloud account

2. Create a database

3. Get your API key, tenant ID, and database name

4. Add to .env file

License

This project is licensed under the MIT License - see the LICENSE file for details.