# Local MLX MCP

An MCP server that turns three locally hosted models into named tools inside Claude Desktop and Codex.

The models run on a Mac mini using MLX. The bridge exposes them over the LAN, so a laptop on the same network can call a local model as a tool without installing anything or shipping the prompt to a hosted API.

## Tools

| Tool | What it does |
| --- | --- |
| `list_local_models` | Lists the models the host is currently serving. |
| `ask_llama` | Llama 3.2 3B. Fast, for quick passes. |
| `ask_qwen` | Qwen 3.5 4B. Reasoning and code. |
| `ask_gemma` | Gemma 4 E2B. General purpose. |

Any of the `ask_*` tools can optionally run a SearXNG search first and pass the results to the model, so a local model can answer questions about things that happened after its training cutoff.

## Why bother

Two reasons, and neither is cost.

The first is data. Some documents cannot go to a hosted API, and "cannot" is a policy fact rather than a preference. Having a local model available as a tool means the sensitive step of a workflow can run locally while the rest of it runs wherever it likes.

The second is that routing is a real design decision. Not every step needs a frontier model. Being able to name a small local model as the worker for a cheap step, in the same interface where the expensive steps run, makes that decision explicit instead of accidental.

## Running it

Over HTTP, for LAN clients:

```bash
node server/mlx-bridge.mjs
# listens on 0.0.0.0:8090
```

Over stdio, for a local client:

```bash
node server/mlx-bridge.mjs --stdio
```

Point it at the host serving the models:

```
MLX_HOST=http://127.0.0.1   # or the host's LAN address
MCP_PORT=8090
```

The upstream model servers are OpenAI-compatible, so the bridge speaks ordinary chat-completions to them on ports 8000, 8001, and 8002.

## Claude Desktop extension

`claude-extension/` packages the same server as a desktop extension with a manifest declaring the four tools. Install it and the models appear as tools without editing a config file by hand.

## Wiring it into a client config

Copy `mcp.json.example`, replace the placeholder paths with absolute paths, and drop it into your client's MCP config location. The example also includes the SearXNG server, since the two are normally used together.

## Running the models at boot

`launchd/` holds the macOS agents that keep the model servers and the bridge running: one per model plus one for the bridge itself. Paths in these files are placeholders and need to be pointed at your install before loading them with `launchctl`.

## Requirements

- Node 18 or newer.
- A host serving MLX models on the expected ports.
- Optional: a SearXNG instance for the search-augmented calls.

Memory is the real constraint. On a 16 GB machine, running three MLX models, an embedding model, and Ollama at once does not work. Pick which ones stay loaded.

## Licence

MIT.