# litellm-mcp

MCP server that lets Claude Code dispatch tasks to the **LiteLLM gateway**
(`192.168.10.175:4000`) as worker agents. Unlike `ollama-mcp` (which reaches only the
local Ollama models directly), this shim reaches the **full gateway model set** — the
local Ollama models *and* the Anthropic-via-gateway models
(`claude-sonnet-4-6`, `claude-haiku-4-5`).

## Tools

- `list_models()` — model ids registered on the gateway.
- `ask_model(prompt, model=None, system=None)` — single-turn worker call. Defaults to
  `qwen2.5:7b-instruct` (a reliable non-reasoning local model — a safer headless default
  than the reasoning model `qwen3.5:4b`).
- `chat(messages, model=None)` — multi-turn.

## Credentials — per-request header (not at rest on Proteus)

This shim brokers spend on Anthropic, so the gateway master key is sensitive. It is **not**
stored in `/etc/mcp/litellm-mcp.env` on the shared Proteus host. Instead the key travels as a
per-request header from the client's `~/.claude.json`:

```json
"litellm": {
    "type": "sse",
    "url": "http://192.168.10.190:8107/sse",
    "headers": { "x-litellm-key": "<LITELLM_MASTER_KEY from Infisical>" }
}
```

The server reads `x-litellm-key` (or `Authorization: Bearer ...`) from each tool call. For
local/stdio use, `LITELLM_API_KEY` env is accepted as a fallback.

## Config (non-sensitive, env file on Proteus)

| Var | Default |
|-----|---------|
| `LITELLM_BASE_URL` | `http://192.168.10.175:4000/v1` |
| `LITELLM_DEFAULT_MODEL` | `qwen2.5:7b-instruct` |
| `MCP_TRANSPORT` / `MCP_HOST` / `MCP_PORT` | `sse` / `0.0.0.0` / `8107` |

## Tests

```bash
pip install -e ".[test]"
pytest -q                       # unit (respx-mocked)
LITELLM_API_KEY=... pytest -q   # also runs live integration against the gateway
```

Integration tests skip automatically when the gateway is unreachable or `LITELLM_API_KEY`
is unset (e.g. in CI on the GitLab runner). Deploy is via GitLab CI → Proteus
(`/opt/mcp/litellm-mcp/`, systemd `litellm-mcp.service`, port 8107).