# local-llm-mcp

English | [日本語](./README.ja.md)

> https://github.com/naka-koma/local-llm-mcp

A thin MCP server for delegating lightweight tasks from Claude Code (or any other MCP-compatible client) to local or cloud LLMs via [LiteLLM](https://www.litellm.ai/).

Designed as a general-purpose tool with no dependency on any specific project. Set it up once, and you can reuse the same configuration from any project on your machine.

## What is this for?

Claude Code's Task tool (the subagent mechanism) only lets you choose between `sonnet`, `opus`, and `haiku` as the model. It isn't designed to let you use other vendors' models (such as local LLMs) directly as subagents.

Instead of "adding more subagents" to work around this limitation, this tool adds "one more tool" to enable delegation to any LLM.

```
Claude Code ──(MCP)──> local-llm-mcp ──(HTTP)──> LiteLLM ──> Ollama (local) / cloud API
```

It's intended for offloading lightweight tasks that the main agent doesn't need to handle itself — summarizing test results, pre-reading logs, simple classification, and so on.

## Prerequisites

- Python 3.10+
- [Ollama](https://ollama.com/) running locally (if using a local model)
- [LiteLLM](https://www.litellm.ai/) running as a proxy

This repository provides only the MCP server itself. Setting up Ollama or LiteLLM is out of scope.

## Setup

### 1. Install dependencies

```bash
pip install -r requirements.txt
```

### 2. Configure environment variables

Refer to `.env.example` and set the LiteLLM endpoint and model name (if you use a `.env` file, you'll need something like `python-dotenv` to load it separately; setting the variables directly in your environment also works).

| Variable | Description | Default |
|---|---|---|
| `LITELLM_BASE_URL` | LiteLLM proxy endpoint | `http://localhost:4000` |
| `LITELLM_MODEL` | The `model_name` registered in LiteLLM's `config.yaml` | `local-gemma` |
| `LITELLM_TIMEOUT_SECONDS` | Response timeout (seconds) | `30` |

### 3. Register with Claude Code

Registering with the `-s user` scope makes the tool available from any project on this machine.

#### Run via uvx (no clone needed — for embedding in other projects)

If [uv](https://docs.astral.sh/uv/) is installed, you can run it directly without cloning the repository. This is the recommended approach when embedding into another project's setup steps (e.g. Flowrite).

```bash
claude mcp add local-llm -s user -- uvx --from git+https://github.com/naka-koma/local-llm-mcp local-llm-mcp
```

On first run, uv automatically builds the environment including dependencies, so `pip install` isn't needed.

#### Use the setup script (recommended for local development)

Runs dependency installation and `claude mcp add` together.

```bash
git clone https://github.com/naka-koma/local-llm-mcp.git
cd local-llm-mcp

# macOS / Linux / Git Bash
bash scripts/setup.sh

# Windows PowerShell
pwsh scripts/setup.ps1
```

#### Register manually

```bash
git clone https://github.com/naka-koma/local-llm-mcp.git
cd local-llm-mcp
pip install -r requirements.txt
claude mcp add local-llm -s user -- python /path/to/local-llm-mcp/server.py
```

On Windows, if `python`'s PATH is unreliable, we recommend using the `py` launcher (`C:\Windows\py.exe`, which has system-wide PATH set up).

```powershell
claude mcp add local-llm -s user -- py D:\path\to\local-llm-mcp\server.py
```

## Provided tool

### `ask_local_llm(prompt: str) -> str`

Sends a prompt to an LLM via LiteLLM and returns the response text. On connection errors or timeouts, it doesn't raise an exception — it returns a string describing the error instead.

## How to use it in your project

We recommend adding a **conditional** instruction like the following to the consuming project's `CLAUDE.md` (or equivalent):

```markdown
## Using a local LLM

If an MCP tool called `local-llm` is available, lightweight tasks such as
summarizing test results or pre-reading logs may be delegated to it.

Even in an environment where the tool doesn't exist, this has no impact
on this project's behavior (delegation is "use it if it's there," not a
required dependency).
```

## License

MIT License. See [LICENSE](./LICENSE) for details.