mcp-llama-swap
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@mcp-llama-swap swap to coder"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
mcp-llama-swap
Hot-swap llama.cpp models inside a running Claude Code session. No context loss. One command.
Plan with a reasoning model. Implement with a coding model. Same session, same context, zero manual overhead.
Supports macOS (launchctl) and Linux (systemd).
Why
Running local LLMs means choosing between a strong reasoning model and a fast coding model. You can't load both on a single machine. Manually swapping models kills your conversation context and flow.
mcp-llama-swap solves this by giving Claude Code a tool to swap the model behind llama-server via your system's service manager (launchctl on macOS, systemd on Linux), while preserving the full conversation history client-side.
Related MCP server: mlx-mcp-server
Quick Start
Install
# Option A: Run directly with uvx (no install needed)
uvx mcp-llama-swap
# Option B: Install from PyPI
pip install mcp-llama-swapConfigure Claude Code
Add to ~/.claude.json:
{
"mcpServers": {
"llama-swap": {
"command": "uvx",
"args": ["mcp-llama-swap"],
"env": {
"LLAMA_SWAP_CONFIG": "/path/to/config.json"
}
}
}
}Configure Models
Create config.json (macOS):
{
"plists_dir": "~/.llama-plists",
"health_url": "http://localhost:8000/health",
"health_timeout": 30,
"models": {
"planner": "qwen35-thinking.plist",
"coder": "qwen3-coder.plist",
"fast": "glm-flash.plist"
}
}Or on Linux:
{
"services_dir": "~/.llama-services",
"health_url": "http://localhost:8000/health",
"health_timeout": 30,
"models": {
"planner": "llama-server-planner.service",
"coder": "llama-server-coder.service"
}
}Use
Inside Claude Code:
You: list models
You: swap to planner
You: <discuss architecture, define interfaces>
You: swap to coder and implement the planThat's it. Context is preserved across swaps.
You can also generate new model configs directly:
You: create a model config named "reasoning" for /models/qwen3-30b.gguf with 8192 contextHow It Works
Claude Code CLI
|
| Anthropic Messages API
v
LiteLLM Proxy (:4000) <-- translates Anthropic -> OpenAI format
|
| OpenAI Chat Completions API
v
llama-server (:8000) <-- model weights swapped via service manager
^
|
mcp-llama-swap <-- this project (launchctl or systemd)Claude Code speaks Anthropic format. LiteLLM translates to OpenAI format for llama-server. This MCP server manages which model service is loaded via launchctl (macOS) or systemd (Linux).
Conversation context survives swaps because Claude Code holds the full message history client-side and re-sends it with every request.
Model Configuration
Mapped Mode (recommended)
Define aliases for your models. Only mapped models are available. Other service configs in the directory are ignored.
macOS:
{
"plists_dir": "~/.llama-plists",
"health_url": "http://localhost:8000/health",
"health_timeout": 30,
"models": {
"planner": "qwen35-35b-a3b-thinking.plist",
"coder": "qwen3-coder.plist",
"fast": "glm-4-7-flash.plist"
}
}Linux:
{
"services_dir": "~/.llama-services",
"health_url": "http://localhost:8000/health",
"health_timeout": 30,
"models": {
"planner": "llama-server-planner.service",
"coder": "llama-server-coder.service"
}
}Swap using your aliases: "swap to coder", "swap to planner".
Directory Mode
Set "models": {} to auto-discover all service configs. Filenames (without extension) become the aliases.
macOS:
{
"plists_dir": "~/.llama-plists",
"models": {}
}Linux:
{
"services_dir": "~/.llama-services",
"models": {}
}MCP Tools
Tool | Description |
| Lists all configured models with load status and current mode |
| Returns the alias of the currently loaded model |
| Unloads current model, loads the specified one, waits for health check |
| Generates a new launchd plist (macOS) or systemd unit (Linux) for a model |
MCP Resources
Resource | Description |
| Current configuration as JSON |
| Current model status, health, and platform info |
MCP Prompts
Prompt | Description |
| Guided plan-then-implement workflow template |
Full Setup Guide
Prerequisites
macOS with launchctl, or Linux with systemd
llama-server (llama.cpp) installed
Model configurations as service files (launchd plists or systemd units)
Python 3.10+
Claude Code CLI pointed at a LiteLLM proxy
1. Install mcp-llama-swap
pip install mcp-llama-swap2. Install and start LiteLLM proxy
pip install litellmCreate litellm_config.yaml:
model_list:
- model_name: "*"
litellm_params:
model: "openai/*"
api_base: "http://localhost:8000/v1"
api_key: "sk-none"
litellm_settings:
drop_params: true
request_timeout: 300Start it:
litellm --config litellm_config.yaml --port 4000On macOS, you can use the included ai.litellm.proxy.plist.template to run it as a persistent launchd service (see setup.sh).
3. Point Claude Code at LiteLLM
Add to ~/.zshrc (macOS) or ~/.bashrc (Linux):
export ANTHROPIC_BASE_URL="http://localhost:4000"
export ANTHROPIC_API_KEY="sk-none"
export ANTHROPIC_MODEL="local"4. Add MCP server to Claude Code
Add to ~/.claude.json:
{
"mcpServers": {
"llama-swap": {
"command": "uvx",
"args": ["mcp-llama-swap"],
"env": {
"LLAMA_SWAP_CONFIG": "/absolute/path/to/config.json"
}
}
}
}5. Create your config.json
Copy config.example.json (macOS) or config.example.linux.json (Linux) and edit with your model aliases and service filenames.
6. Create model service configs
You can create service configs manually, or use the create_model_config MCP tool inside Claude Code:
You: create a model config named "coder" for /path/to/model.gguf with 8192 contextThis generates the appropriate launchd plist (macOS) or systemd unit file (Linux) in your services directory.
Automated Setup (macOS)
If you prefer a one-shot setup on macOS, clone this repo and run:
git clone https://github.com/oussama-kh/mcp-llama-swap.git ~/mcp-llama-swap
cd ~/mcp-llama-swap
chmod +x setup.sh
./setup.shThe script creates a virtual environment, installs dependencies, configures the LiteLLM launchd service, and prints the exact config to add.
Configuration Reference
config.json fields:
Field | Default | Description |
|
| Directory containing model service configs |
| — | macOS alias for |
| — | Linux alias for |
|
| llama-server health endpoint |
|
| Seconds to wait for health check after loading |
|
| Alias-to-filename map. Empty = directory mode |
|
| Service manager: |
|
| macOS only: |
Override config path via the LLAMA_SWAP_CONFIG environment variable.
Platform Details
macOS (launchctl)
Models are managed as launchd services via plist files. Two launchctl modes are available:
Legacy (default): Uses
launchctl load/unload/list. Works on all macOS versions.Modern: Uses
launchctl bootstrap/bootout/print. The officially supported API on newer macOS. Enable with"launchctl_mode": "modern"in config.
Linux (systemd)
Models are managed as systemd user services. Unit files in services_dir are symlinked to ~/.config/systemd/user/ and managed via systemctl --user start/stop.
Troubleshooting
LiteLLM not translating correctly: Check /tmp/litellm.stderr.log. Verify llama-server is running: curl http://localhost:8000/health.
Model swap times out: Increase health_timeout in config.json. Large models may need 30+ seconds to load weights into memory.
Claude Code cannot find the MCP server: Verify the LLAMA_SWAP_CONFIG path is absolute. Test directly: python -m mcp_llama_swap.
Mapped model not found: The service filename in models must match an actual file in your services directory.
systemd service won't start: Check journalctl --user -u llama-server-<name> for errors. Ensure llama-server is in your PATH.
launchctl modern mode issues: If bootstrap/bootout commands fail, fall back to "launchctl_mode": "legacy" in config.
Development
# Install with test dependencies
pip install -e ".[test]"
# Run tests
pytest -vUse Case
This project enables a two-phase AI coding workflow entirely on local hardware:
Planning phase: Load a reasoning model (e.g., Qwen3.5-35B-A3B with thinking). Discuss architecture, define interfaces, decompose requirements.
Implementation phase: Swap to a coding model (e.g., Qwen3-Coder-30B). Execute the plan file by file with full conversation context from the planning phase.
No cloud APIs. No data leaving your machine. No context loss between phases.
License
Apache-2.0
Available Tools
4 toolscreate_model_configA
Generate a new service config for a llama-server model.
Creates a launchd plist (macOS) or systemd unit (Linux) that can be used with swap_model.
Args: name: Short alias for the model (e.g., "coder", "planner") model_path: Absolute path to the GGUF model file context_size: Context window size (default: 4096) gpu_layers: Number of GPU layers, -1 for all (default: -1) port: Port for llama-server (default: 8000)
Inputs: name (required), model_path (required), context_size, gpu_layers, +1 more. Returns: result
get_current_modelA
Get the currently loaded llama.cpp model.
Inputs: no named parameters. Returns: result
list_modelsA
List available llama.cpp model configurations and their load status.
Inputs: no named parameters. Returns: result
swap_modelA
Swap to a different llama.cpp model.
Unloads any currently loaded model, loads the requested one, and waits for the health endpoint to confirm readiness.
Args: model: Alias of the model to load
Inputs: model (required). Returns: result
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 4 tools
Each tool has a clearly distinct function: creating configs, listing models, getting the current model, and swapping to a different model. No overlap exists.
All tool names follow a consistent verb_noun pattern in snake_case (create_model_config, get_current_model, list_models, swap_model), making them predictable.
Four tools is well-scoped for the server's purpose of managing llama.cpp model configurations and swapping. Each tool serves a necessary role without redundancy.
The set covers the key operations: creating a config, listing models, checking the current model, and swapping. However, there is no tool to delete or update a model config, which is a minor gap.
Maintenance
Related MCP Connectors
Shared long-term memory & Stop-hook task handoff across Claude Code, Codex, Cursor, and mobile.
141Share context and questions between Claude instances — VS Code, claude.ai web, and mobile.
Shared notes, logs, issues and secrets for Claude, ChatGPT and local models.
Capture coding-agent sessions and resume them in any other agent: Claude Code, Codex, Cursor, more.
Related MCP Servers
- AlicenseCqualityDmaintenanceBridges Claude Desktop with local LLM instances running via llama-server, enabling full conversation support with complete parameter control and health monitoring. Allows users to chat with their local models directly through Claude Desktop with configurable sampling parameters.313 npm10Creative Commons Zero v1.0 Universal
- AlicenseAqualityBmaintenanceBridges Claude Code and Claude Desktop to a locally-running LLM, optimized for MLX LM on Apple Silicon with support for any OpenAI-compatible backend.423 PyPI1MIT
- AlicenseAqualityDmaintenanceEnables LLMs like Claude to bidirectionally chat with diffusion-based LLMs (e.g., DiffusionGemma, LLaDA) via a persistent llama-diffusion-cli process, with support for restarting sessions and full configuration.1MIT
- AlicenseAqualityFmaintenanceMCP server that routes Claude Desktop chat into Claude Code, enabling headless execution and terminal handoff with session continuity.4MIT