# LFM2.5 — On-Device AI Chat & MCP Server

Run [Liquid AI's LFM2.5-1.2B-Instruct](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-MLX-8bit) model locally on your Mac with a beautiful chat interface and MCP integration for tools like Claude Desktop, Cursor, and OpenClaw.

<div align="center">

**~120 tok/s on Apple Silicon** · **~480 tok/s across 4 Macs** · **1.5 GB RAM** · **100% Private & Offline**

</div>

## ✨ Features

- 🖥️ **Beautiful Chat UI** — Dark-themed, responsive chat interface with markdown rendering, code highlighting, and real-time performance metrics
- ⚡ **MLX Acceleration** — Runs natively on Apple Silicon via MLX with 8-bit quantization
- 🔌 **MCP Server** — Expose LFM2.5 as tools for Claude Desktop, Cursor, OpenClaw, and any MCP-compatible client
- 🔀 **Multi-Mac Cluster** — Distribute inference across multiple M4 Macs with automatic load balancing and failover
- 📊 **Live Stats** — Token count, generation speed (tok/s), and response time displayed in real-time
- 🎯 **Quick Prompts** — Pre-built prompts for common tasks (CRISPR explanation, code architecture, creative writing, data analysis)

## 🏗️ Architecture

```
Single-Mac Mode:
  Client → MLX Server (:8080)

Cluster Mode (4× throughput):
  Client → Cluster LB (:5200) → Mac 1 MLX (:8080)
                               → Mac 2 MLX (:8080)
                               → Mac 3 MLX (:8080)
                               → Mac 4 MLX (:8080)
```

## 🚀 Quick Start

### Prerequisites

- macOS with Apple Silicon (M1/M2/M3/M4)
- Python 3.10+
- [MLX](https://github.com/ml-explore/mlx) and [mlx-lm](https://github.com/ml-explore/mlx-examples/tree/main/llms)

### 1. Install

```bash
# Clone the repo
git clone https://github.com/WispAyr/LFM2.5-local.git
cd LFM2.5-local

# Install MLX dependencies (if needed)
pip install mlx-lm

# Install MCP dependencies
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

### 2. Run

```bash
# Just the MLX server + chat UI
./start_server.sh

# MLX + MCP server (for Claude Desktop, Cursor, OpenClaw)
./start_server.sh --mcp

# Everything (MLX + MCP + web UI)
./start_server.sh --all

# Multi-Mac cluster (MLX + Load Balancer + MCP + web UI)
./start_server.sh --cluster
```

### 3. Open

| Service | URL | Description |
|---------|-----|-------------|
| Chat UI | http://localhost:3000 | Web chat interface |
| MLX API | http://localhost:8080/v1 | OpenAI-compatible API |
| Cluster LB | http://localhost:5200/v1 | Load-balanced API (cluster mode) |
| Cluster Status | http://localhost:5200/cluster/status | Live node health |
| MCP Server | http://localhost:5100/mcp | MCP protocol endpoint |

## 🔌 MCP Integration

The MCP server exposes LFM2.5 as tools that any MCP-compatible client can call.

### Tools

| Tool | Description |
|------|-------------|
| `chat` | General-purpose chat — questions, reasoning, writing |
| `summarize` | Summarize text with configurable length |
| `analyze_code` | Code review, bug detection, suggestions |
| `translate` | Translate text to any language |

### Resources

| URI | Description |
|-----|-------------|
| `lfm25://model/info` | Model architecture, specs, capabilities |
| `lfm25://server/status` | Live MLX server connection status |

### Prompts

| Prompt | Description |
|--------|-------------|
| `code_review` | Structured code review template |
| `explain_concept` | ELI5-style concept explanation |

### Connect to Claude Desktop

Add to `~/Library/Application Support/Claude/claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "lfm25": {
      "command": "/path/to/LFM2.5-local/.venv/bin/python3",
      "args": ["/path/to/LFM2.5-local/mcp_server.py"]
    }
  }
}
```

### Connect to Cursor

Settings → MCP Servers → Add and paste the same config.

### Connect to OpenClaw

Point to `http://localhost:5100/mcp` as an MCP server endpoint.

## 📁 Project Structure

```
├── index.html            # Chat web UI (single-file, no build step)
├── mcp_server.py         # FastMCP server with tools, resources, prompts
├── cluster.py            # Multi-Mac cluster load balancer
├── cluster_config.json   # Cluster node configuration
├── server.py             # Web UI proxy server (serves static + proxies API)
├── mcp_config.json       # MCP config template for Claude Desktop / Cursor
├── start_server.sh       # Launcher script (--mcp, --all, --cluster flags)
├── demo_lfm25.py         # Python demo script
└── requirements.txt      # Python dependencies
```

## 🔀 Multi-Mac Cluster

Distribute inference across multiple M4 Macs for **~480 tok/s combined throughput** and automatic failover.

### Setup

**1. On each Mac**, start the MLX server:
```bash
pip install mlx-lm
python3 -m mlx_lm.server --model LiquidAI/LFM2.5-1.2B-Instruct-MLX-8bit --port 8080 --host 0.0.0.0
```

**2. Edit `cluster_config.json`** on your primary Mac with each node's IP:
```json
{
  "nodes": [
    {"name": "Mac-1", "host": "192.168.1.10", "port": 8080, "weight": 1},
    {"name": "Mac-2", "host": "192.168.1.11", "port": 8080, "weight": 1},
    {"name": "Mac-3", "host": "192.168.1.12", "port": 8080, "weight": 1},
    {"name": "Mac-4", "host": "192.168.1.13", "port": 8080, "weight": 1}
  ]
}
```

**3. Start the cluster:**
```bash
./start_server.sh --cluster
```

### Features

- **Least-loaded routing** for POST requests, round-robin for GETs
- **Health checking** every 5 seconds with automatic failover
- **SSE streaming** passthrough for real-time token generation
- **Node weights** for prioritizing faster Macs
- **Status dashboard** at `/cluster/status` with per-node metrics

## 🧠 About LFM2.5

LFM2.5 is **not a Transformer**. It's a hybrid architecture from [Liquid AI](https://liquid.ai) combining:

- **10× Gated Short Convolution** layers — fast sequential processing
- **6× Grouped Query Attention** layers — selective attention

This gives it a unique performance profile: extremely fast inference (~120+ tok/s on M-series Macs) with a small 1.17B parameter footprint.

| Spec | Value |
|------|-------|
| Parameters | 1.17B |
| Layers | 16 |
| Context | 32K tokens |
| Vocab | 65,536 |
| Training | 28T tokens |
| Quantization | 8-bit (MLX) |

## 📄 License

MIT

## 🔗 Links

- [Liquid AI](https://liquid.ai)
- [LFM2.5 on HuggingFace](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-MLX-8bit)
- [LFM2.5 Paper](https://arxiv.org/abs/2412.04332)
- [MLX](https://github.com/ml-explore/mlx)
- [FastMCP](https://gofastmcp.com)