LFM2.5 — On-Device AI Chat & MCP Server

Run Liquid AI's LFM2.5-1.2B-Instruct model locally on your Mac with a beautiful chat interface and MCP integration for tools like Claude Desktop, Cursor, and OpenClaw.

~120 tok/s on Apple Silicon · ~480 tok/s across 4 Macs · 1.5 GB RAM · 100% Private & Offline

✨ Features

• 🖥️ Beautiful Chat UI — Dark-themed, responsive chat interface with markdown rendering, code highlighting, and real-time performance metrics

• ⚡ MLX Acceleration — Runs natively on Apple Silicon via MLX with 8-bit quantization

• 🔌 MCP Server — Expose LFM2.5 as tools for Claude Desktop, Cursor, OpenClaw, and any MCP-compatible client

• 🔀 Multi-Mac Cluster — Distribute inference across multiple M4 Macs with automatic load balancing and failover

• 📊 Live Stats — Token count, generation speed (tok/s), and response time displayed in real-time

• 🎯 Quick Prompts — Pre-built prompts for common tasks (CRISPR explanation, code architecture, creative writing, data analysis)

Related MCP server: local-mmcp

🏗️ Architecture

Single-Mac Mode:
  Client → MLX Server (:8080)

Cluster Mode (4× throughput):
  Client → Cluster LB (:5200) → Mac 1 MLX (:8080)
                               → Mac 2 MLX (:8080)
                               → Mac 3 MLX (:8080)
                               → Mac 4 MLX (:8080)

🚀 Quick Start

Prerequisites

• macOS with Apple Silicon (M1/M2/M3/M4)

• Python 3.10+

• MLX and mlx-lm

1. Install

# Clone the repo
git clone https://github.com/WispAyr/LFM2.5-local.git
cd LFM2.5-local

# Install MLX dependencies (if needed)
pip install mlx-lm

# Install MCP dependencies
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

2. Run

# Just the MLX server + chat UI
./start_server.sh

# MLX + MCP server (for Claude Desktop, Cursor, OpenClaw)
./start_server.sh --mcp

# Everything (MLX + MCP + web UI)
./start_server.sh --all

# Multi-Mac cluster (MLX + Load Balancer + MCP + web UI)
./start_server.sh --cluster

3. Open

🔌 MCP Integration

The MCP server exposes LFM2.5 as tools that any MCP-compatible client can call.

Tools

Resources

Prompts

Connect to Claude Desktop

Add to ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "lfm25": {
      "command": "/path/to/LFM2.5-local/.venv/bin/python3",
      "args": ["/path/to/LFM2.5-local/mcp_server.py"]
    }
  }
}

Connect to Cursor

Settings → MCP Servers → Add and paste the same config.

Connect to OpenClaw

Point to http://localhost:5100/mcp as an MCP server endpoint.

📁 Project Structure

├── index.html            # Chat web UI (single-file, no build step)
├── mcp_server.py         # FastMCP server with tools, resources, prompts
├── cluster.py            # Multi-Mac cluster load balancer
├── cluster_config.json   # Cluster node configuration
├── server.py             # Web UI proxy server (serves static + proxies API)
├── mcp_config.json       # MCP config template for Claude Desktop / Cursor
├── start_server.sh       # Launcher script (--mcp, --all, --cluster flags)
├── demo_lfm25.py         # Python demo script
└── requirements.txt      # Python dependencies

🔀 Multi-Mac Cluster

Distribute inference across multiple M4 Macs for ~480 tok/s combined throughput and automatic failover.

Setup

1. On each Mac, start the MLX server:

pip install mlx-lm
python3 -m mlx_lm.server --model LiquidAI/LFM2.5-1.2B-Instruct-MLX-8bit --port 8080 --host 0.0.0.0

2. Edit cluster_config.json on your primary Mac with each node's IP:

{
  "nodes": [
    {"name": "Mac-1", "host": "192.168.1.10", "port": 8080, "weight": 1},
    {"name": "Mac-2", "host": "192.168.1.11", "port": 8080, "weight": 1},
    {"name": "Mac-3", "host": "192.168.1.12", "port": 8080, "weight": 1},
    {"name": "Mac-4", "host": "192.168.1.13", "port": 8080, "weight": 1}
  ]
}

3. Start the cluster:

./start_server.sh --cluster

Features

• Least-loaded routing for POST requests, round-robin for GETs

• Health checking every 5 seconds with automatic failover

• SSE streaming passthrough for real-time token generation

• Node weights for prioritizing faster Macs

• Status dashboard at /cluster/status with per-node metrics

🧠 About LFM2.5

LFM2.5 is not a Transformer. It's a hybrid architecture from Liquid AI combining:

• 10× Gated Short Convolution layers — fast sequential processing

• 6× Grouped Query Attention layers — selective attention

This gives it a unique performance profile: extremely fast inference (~120+ tok/s on M-series Macs) with a small 1.17B parameter footprint.

📄 License

MIT

🔗 Links

• Liquid AI

• LFM2.5 on HuggingFace

• LFM2.5 Paper

• MLX

• FastMCP