MCP Chat

Note: This project is an extension of the MCP certification project originally sourced from the Anthropic Claude MCP course. It has been extended to support local LLMs instead of the Anthropic API, along with additional tooling and improvements.

A command-line interface application that enables interactive chat with a local LLM (via Ollama or any OpenAI-compatible server) using the MCP (Model Context Protocol) architecture for document management.

Prerequisites

• Python 3.9+

• Ollama or any OpenAI-compatible local LLM server

Setup

Step 1: Configure environment variables

Create a .env file in the project root:

LOCAL_LLM_MODEL=llama3.2
LOCAL_LLM_BASE_URL=http://localhost:11434/v1
USE_UV=1   # Set to 0 if not using uv

Step 2: Install dependencies

Option 1: With uv (Recommended)

uv is a fast Python package installer and resolver.

pip install uv
uv venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
uv pip install -e .

Option 2: Without uv

python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install openai python-dotenv prompt-toolkit "mcp[cli]==1.8.0"

Step 3: Start your local LLM

# Ollama example
ollama serve
ollama pull llama3.2

Step 4: Run the project

# With uv
uv run main.py

# With additional MCP servers
uv run main.py extra_server.py another_server.py

Usage

Basic Chat

Type your message and press Enter:

> What is the state of the condenser tower?

Document Retrieval

Use @ followed by a document ID to include its contents in your query:

> Tell me about @deposition.md
> Summarize @financials.docx

Demo

MCP Chat running with llama3.2 via Ollama on Windows

Commands

Use / prefix to execute MCP prompts. Press Tab to autocomplete:

> /format deposition.md
> /summarize report.pdf

Available Documents

Architecture

main.py
  ├── MCPClient          # Manages stdio communication with MCP server(s)
  ├── mcp_server.py      # FastMCP server — tools, resources, prompts
  ├── core/claude.py     # Local LLM integration (OpenAI-compatible)
  ├── core/cli_chat.py   # Chat logic with @ and / command handling
  └── core/cli.py        # Terminal UI with Tab autocomplete

MCP Server Features

Development

Adding New Documents

Edit the docs dictionary in mcp_server.py:

docs = {
    "your_doc.md": "Your document content here",
}

Adding New MCP Servers

Pass additional server scripts as arguments when running:

uv run main.py your_custom_server.py

Adding New Tools / Prompts / Resources

Use the FastMCP decorators in mcp_server.py:

@mcp.tool(name="my_tool", description="Does something useful")
def my_tool(input: str) -> str:
    return f"Processed: {input}"

Known Limitations

• Document edits are in-memory only and lost on server restart

• No persistent storage backend

• No authentication or multi-user support (stdio only — single client per server instance)

Troubleshooting

OneDrive hardlink error on Windows:

$env:UV_LINK_MODE = "copy"; uv pip install -e .

Local LLM not responding:

• Ensure Ollama is running: ollama serve

• Confirm the model is pulled: ollama pull llama3.2

• Check LOCAL_LLM_BASE_URL in .env matches your server

Module not found errors:

uv add openai python-dotenv mcp