# llm-chat-mcp

MCP server for interacting with LLM models configured in Continue.dev's `config.yaml`.

## What it does

Exposes 4 tools that let the agent inspect your configured models and send chat requests to them:

- **`llm_chat_list_models`** — list all models in your config with their names and ids
- **`llm_chat_get_model_params`** — inspect parameters (temperature, topP, etc.) for any model
- **`llm_chat_get_model_prompt`** — read the system prompt configured for a model
- **`llm_chat_send_request`** — chat with a model, optionally overriding parameters or loading prompts from files

## Installation

```bash
pip install -e .
```

Or run directly from source without install.

## CLI Usage

```bash
python -m llm_chat_mcp --default-model "GLM-5.2-FP8"
python -m llm_chat_mcp --config /path/to/config.yaml --default-model "ModelName"
python -m llm_chat_mcp --help
```

| Argument | Description | Default |
|----------|-------------|---------|
| `--config PATH` | Path to config.yaml | `~/.continue/config.yaml` |
| `--default-model NAME` | Default model for send_request | *(none)* |
| `--timeout SECONDS` | Request timeout in seconds | `19` |
| `--relative_paths_base PATH` | Base directory for resolving relative output file paths | *(process cwd)* |
| `--auto-output-dir PATH` | Directory for auto-generated output files when response exceeds `auto_file_threshold` and no `output_file_path` is set | *(OS temp directory)* |

## Configuration

- Config is re-read on every request — no restart needed
- Auth uses `apiKey` from each model entry in config.yaml
- If no model is specified (neither in CLI nor tool call), an error tells you how to set one

## `llm_chat_send_request` parameters

The main tool. Sends a chat completion request to an LLM and returns the response.

### Input parameters

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `model_selector` | str | *(CLI default)* | Model name or id from config.yaml |
| `prompt_text` | str | *(none)* | Prompt text to send |
| `prompt_files` | list | *(none)* | List of file specs (string path or `{path, start_line, end_line}` dict) |
| `include_line_numbers` | bool | `true` | Prefix each file line with `N: ` |
| `system_prompt` | str | *(from config)* | Override system message; `""` suppresses entirely |
| `temperature`, `topP`, `topK`, `minP` | float | *(from config)* | Sampling parameters |
| `maxTokens`, `presencePenalty`, `frequencyPenalty` | int/float | *(from config)* | Generation parameters |
| `extraParams` | dict | *(none)* | Extra body properties merged into API request |
| `details` | bool | `false` | Include reasoning/thinking content in response |
| `timeout` | float | *(CLI default)* | Per-request timeout override |
| `output_file_path` | str | *(none)* | Write full response to this file (relative paths resolved against `--relative_paths_base`) |
| `append` | bool | `false` | Append to `output_file_path` instead of overwriting; returns `appended_line_start`/`appended_line_end` |
| `inline_preview_chars` | int | `500` | Max chars of content returned inline; preview applies only when a file is written |
| `auto_file_threshold` | int | `8000` | Auto-write response to file when `response_chars` exceeds this and no `output_file_path` is set; `0` disables |

### Response structure

Always returned as JSON:

```json
{
  "content": "<preview, full content, or empty>",
  "truncated": true,
  "metadata": {
    "model_name": "...",
    "model": "...",
    "elapsed_seconds": 1.23,
    "request_sent": {...},
    "response_headers": {...},
    "response_chars": 1234,
    "output_file": "...",
    "auto_output_file": "...",
    "created_dirs": [...],
    "appended_line_start": 201,
    "appended_line_end": 250
  }
}
```

### Output strategy

The tool chooses one of three strategies based on parameters and response size:

1. **Explicit file** (`output_file_path` set): full response written to the file. If `append=true`, the response is appended and `appended_line_start`/`appended_line_end` (1-based, inclusive) are returned so the caller can read only the appended slice via `extract_lines`.
2. **Auto file** (no `output_file_path`, `auto_file_threshold > 0`, `response_chars > threshold`): full response written to an auto-generated file in `--auto-output-dir` (or OS temp). Filename format: `llm_output_<YYYYMMDD_HHMMSS>_<6-char-uuid>.json`.
3. **Inline only** (no file written): full content returned in the `content` field.

### Inline preview

When a file is written (explicit or auto), the `content` field contains a preview of the response:

- If `inline_preview_chars > 0` and `len(content) > inline_preview_chars`: truncated preview with suffix `[truncated, full response in <file_path>]`.
- If `inline_preview_chars > 0` and `len(content) <= inline_preview_chars`: full content (fits in preview).
- If `inline_preview_chars == 0`: empty `content` (file has the full response).

When no file is written, the full content is returned inline regardless of `inline_preview_chars` — this prevents data loss.

## Continue.dev Integration

Add to `.continue/mcpServers/llm-chat.yaml`:

```yaml
name: LLM Chat MCP server
version: 0.2.0
schema: v1
mcpServers:
  - name: LLM Chat MCP server
    command: python
    args:
      - "-m"
      - "llm_chat_mcp"
      - "--default-model"
      - "GLM-5.2-FP8"
      - "--timeout"
      - "570"
      - "--relative_paths_base"
      - "/path/to/your/workspace"
      - "--auto-output-dir"
      - "/path/to/your/workspace/.continue/skills/large-tasks/tmp-outputs"
    env:
      PYTHONPATH: "/path/to/llm-chat-mcp"
```

Then reload Continue.dev.

## Project Structure

```
llm-chat-mcp/
├── pyproject.toml          # Dependencies: mcp, pyyaml, httpx
├── README.md               # This file
├── llm_chat_mcp/
│   ├── __init__.py
│   ├── __main__.py         # CLI entry point + tool registration
│   ├── config.py           # Config loading, model resolution
│   └── api.py              # API client, error handling
└── tests/
    └── test_output_strategies.py  # Tests for append, inline_preview, auto_file
```

## Testing

```bash
python tests/test_output_strategies.py
```

Tests mock the API call and verify the file-writing and response assembly logic. Covers all combinations of `output_file_path`, `append`, `inline_preview_chars`, and `auto_file_threshold`.