uv run python -m client.src.main "Explain how MCP tools work with RAG"

uv run python -m client.src.main --interactive

MCP RAG Agent

Type a question and press Enter.
Type 'exit' or 'quit' to stop.

You: Explain how MCP tools work with RAG

Assistant:
MCP tools allow the agent to access external capabilities, such as document retrieval, through a standard protocol...

Sources:
- knowledge/mcp.md#3
- knowledge/rag.md#1

flowchart TD
    User[User] --> Agent[LangGraph Agent]
    Agent --> Runtime[Agent Runtime]

    Runtime --> LLM["LLM Provider<br/>(Ollama / Bedrock)"]
    Runtime --> MCPClient[MCP Client]

    MCPClient --> MCPServer[MCP Server]

    MCPServer --> RetrievalTool["retrieve_documents()"]
    MCPServer --> OtherTools["Other MCP Tools"]

    RetrievalTool --> Embed["Embedding Provider<br/>(Ollama / Bedrock)"]
    Embed --> VectorStore[ChromaDB Vector Store]
    VectorStore --> Reranker[Reranker]
    Reranker --> Context[Context Builder]

    Context --> RetrievalTool
    RetrievalTool --> MCPServer
    MCPServer --> MCPClient
    Runtime --> Answer[Grounded Answer with Citations]

[knowledge/mcp.md#2]

client/
└── src/
    ├── main.py                 # CLI entry point
    ├── agent_runtime.py        # Agent loop and tool-calling runtime
    ├── mcp_client.py           # MCP client wrapper
    ├── tool_executor.py        # Executes MCP tools
    ├── tool_mapper.py          # Converts MCP tools to Ollama tool schema
    └── llm/                    # LLM provider implementations
        ├── base.py
        ├── ollama.py
        └── bedrock.py
        
server/
└── src/
    ├── server.py               # MCP server entry point
    └── tools/                  # MCP tool registrations

services/
├── rag/
├── chunking/                   # Text chunking strategies
├── dto/                        # API response DTOs
├── embedding/                  # Embedding providers
|      ├── base.py
|      ├── ollama_provider.py
|      └── bedrock_provider.py
├── loaders/                    # Document loaders
├── mappers/                    # Domain-to-response mappers
├── models/                     # RAG domain models
├── pipelines/                  # Indexing and retrieval pipelines
├── rerankers/                  # Reranking implementations
└── vectorstores/               # Vector store adapters

knowledge/                      # Local knowledge base documents
scripts/                        # Utility scripts
chroma_db/                      # Local ChromaDB data

ollama pull llama3.1

uv sync

ollama serve

uv run python -m client.src.main "Explain how MCP tools work with RAG"

uv run python -m client.src.main --interactive

uv run python -m client.src.main --help

question              Optional question to ask the agent
-i, --interactive     Start an interactive chat session
--model               Ollama chat model to use
--max-steps           Maximum number of tool-calling steps

uv run python -m client.src.main "What is AWS Bedrock?" --model llama3.1

uv run python -m client.src.main --interactive --max-steps 8

{
  "context": "...",
  "sources": [
    {
      "document_id": "knowledge/mcp.md",
      "chunk_index": 2
    }
  ]
}

[document_id#chunk_index]

MCP lets an agent call external tools through a standard protocol, which makes the RAG system easier to separate from the LLM runtime [knowledge/mcp.md#2].

Sources:
- knowledge/mcp.md#2
- knowledge/rag.md#4

retrieve_documents(query: str, top_k: int = 20, top_n: int = 5)

uv run python -m scripts.evaluate_retrieval

uv run python -m scripts.evaluate_retrieval --top-k 8 --top-n 4

uv run python -m scripts.evaluate_retrieval --json-output scripts/eval/results/latest.json

 uv add fastapi uvicorn

uv run uvicorn api.main:app --reload

[http://localhost:8000](http://localhost:8000)

[http://localhost:8000/docs](http://localhost:8000/docs)

curl [http://localhost:8000/health](http://localhost:8000/health)

curl -X POST [http://localhost:8000/chat](http://localhost:8000/chat)
-H "Content-Type: application/json"
-d '{"message": "Explain how MCP tools work with RAG"}'

curl -N -X POST [http://localhost:8000/chat/stream](http://localhost:8000/chat/stream)
-H "Content-Type: application/json"
-d '{"message": "Explain how MCP tools work with RAG"}'

curl -X POST [http://localhost:8000/retrieve](http://localhost:8000/retrieve)
-H "Content-Type: application/json"
-d '{"query": "What is Model Context Protocol?", "top_k": 8, "top_n": 4}'

curl [http://localhost:8000/tools](http://localhost:8000/tools)

cp .env.example .env

CLIENT_LLM_PROVIDER=ollama OLLAMA_CHAT_MODEL=llama3.1 RAG_EMBEDDING_PROVIDER=ollama OLLAMA_EMBEDDING_MODEL=nomic-embed-text

CLIENT_LLM_PROVIDER=bedrock BEDROCK_CHAT_MODEL=anthropic.claude-3-5-sonnet-20240620-v1:0 AWS_REGION=us-east-1
RAG_EMBEDDING_PROVIDER=ollama OLLAMA_EMBEDDING_MODEL=nomic-embed-text

CLIENT_LLM_PROVIDER=bedrock BEDROCK_CHAT_MODEL=anthropic.claude-3-5-sonnet-20240620-v1:0
RAG_EMBEDDING_PROVIDER=bedrock BEDROCK_EMBEDDING_MODEL=amazon.titan-embed-text-v2:0
AWS_REGION=us-east-1