# MCP Chat

MCP Chat is a command-line interface application that enables interactive chat capabilities with AI models. It can run against either the hosted Anthropic API **or a fully local model** (Ollama via a LiteLLM proxy). The application supports document retrieval, command-based prompts, and extensible tool integrations via the MCP (Model Control Protocol) architecture.

## Prerequisites

- Python 3.9+
- Either an Anthropic API key (hosted) **or** Ollama installed (local, no key required)

## Running with a local model (no API key)

The app talks to the Anthropic SDK; a small [LiteLLM](https://github.com/BerriAI/litellm) proxy translates that to a local model, so no code changes are needed to switch backends — only `.env`. Two local backends are supported (defined in `litellm_config.yaml`): **llama.cpp** (default) and **Ollama**. Both serve the same `qwen2.5:14b` weights.

Common to both:

- `.env` is already set for local use:
  ```
  ANTHROPIC_BASE_URL="http://localhost:4000"
  ANTHROPIC_API_KEY="not-needed"
  ```
- Start the LiteLLM proxy in its own terminal and leave it running:
  ```bash
  uv run litellm --config litellm_config.yaml
  ```
- Run the app in another terminal:
  ```bash
  uv run main.py
  ```

> Memory note: a 14B model at Q4 (~9 GB) runs comfortably on 24 GB+ of RAM. On 16–18 GB it works but is memory-tight — only one 14B can be GPU-resident at a time, so don't run llama.cpp and Ollama simultaneously. For lighter machines use a 7B variant.

### Backend A — llama.cpp (default: `CLAUDE_MODEL="qwen2.5-llamacpp"`)

1. Install: `brew install llama.cpp`
2. Get the weights. If you've already pulled the model with Ollama (below), llama.cpp can load that exact GGUF blob — no second download. Otherwise download a `qwen2.5-14b-instruct` GGUF from Hugging Face.
3. Start `llama-server` (leave running). Flash-attention + quantized KV cache are **required** to fit a 14B on ~18 GB:
   ```bash
   llama-server \
     -m ~/.ollama/models/blobs/sha256-2049f5674b1e92b4464e5729975c9689fcfbf0b0e4443ccf10b5339f370f9a54 \
     --jinja -c 4096 -np 1 -fa on \
     --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 -ngl 999 \
     --host 127.0.0.1 --port 8080 --alias qwen2.5-14b
   ```
   `--jinja` enables tool calling (the app relies on it). The blob path is the GGUF Ollama stored; check yours with `ls -lhS ~/.ollama/models/blobs/`.

### Backend B — Ollama (set `CLAUDE_MODEL="qwen2.5-local"`)

1. Install and start: `brew install ollama && brew services start ollama`
2. Download the model: `ollama pull qwen2.5:14b`

Ollama runs as a background service (no separate terminal needed) and handles flash-attention/KV settings itself.

## Running with the hosted Anthropic API

In `.env`, comment out `ANTHROPIC_BASE_URL`, set a real key, and use a real model id:

```
CLAUDE_MODEL="claude-sonnet-4-5"
ANTHROPIC_API_KEY="sk-ant-..."
```

Then `uv run main.py` (no proxy needed).

## Setup

### Step 1: Configure the environment variables

1. Create or edit the `.env` file in the project root and verify that the following variables are set correctly:

```
ANTHROPIC_API_KEY=""  # Enter your Anthropic API secret key
```

### Step 2: Install dependencies

#### Option 1: Setup with uv (Recommended)

[uv](https://github.com/astral-sh/uv) is a fast Python package installer and resolver.

1. Install uv, if not already installed:

```bash
pip install uv
```

2. Create and activate a virtual environment:

```bash
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
```

3. Install dependencies:

```bash
uv pip install -e .
```

4. Run the project

```bash
uv run main.py
```

#### Option 2: Setup without uv

1. Create and activate a virtual environment:

```bash
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
```

2. Install dependencies:

```bash
pip install anthropic python-dotenv prompt-toolkit "mcp[cli]==1.8.0"
```

3. Run the project

```bash
python main.py
```

## Usage

### Basic Interaction

Simply type your message and press Enter to chat with the model.

### Document Retrieval

Use the @ symbol followed by a document ID to include document content in your query:

```
> Tell me about @deposition.md
```

### Commands

Use the / prefix to execute commands defined in the MCP server:

```
> /summarize deposition.md
```

Commands will auto-complete when you press Tab.

## Development

### Adding New Documents

Edit the `mcp_server.py` file to add new documents to the `docs` dictionary.

### Implementing MCP Features

To fully implement the MCP features:

1. Complete the TODOs in `mcp_server.py`
2. Implement the missing functionality in `mcp_client.py`

### Linting and Typing Check

There are no lint or type checks implemented.