Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@mcp-llama-swap swap to coder"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Hot-swap llama.cpp models inside a running Claude Code session. No context loss. One command.
Plan with a reasoning model. Implement with a coding model. Same session, same context, zero manual overhead.
Supports macOS (launchctl) and Linux (systemd).
Running local LLMs means choosing between a strong reasoning model and a fast coding model. You can't load both on a single machine. Manually swapping models kills your conversation context and flow.
mcp-llama-swap solves this by giving Claude Code a tool to swap the model behind llama-server via your system's service manager (launchctl on macOS, systemd on Linux), while preserving the full conversation history client-side.
Related MCP server: mlx-mcp-server
# Option A: Run directly with uvx (no install needed)
uvx mcp-llama-swap
# Option B: Install from PyPI
pip install mcp-llama-swapAdd to ~/.claude.json:
{
"mcpServers": {
"llama-swap": {
"command": "uvx",
"args": ["mcp-llama-swap"],
"env": {
"LLAMA_SWAP_CONFIG": "/path/to/config.json"
}
}
}
}Create config.json (macOS):
{
"plists_dir": "~/.llama-plists",
"health_url": "http://localhost:8000/health",
"health_timeout": 30,
"models": {
"planner": "qwen35-thinking.plist",
"coder": "qwen3-coder.plist",
"fast": "glm-flash.plist"
}
}Or on Linux:
{
"services_dir": "~/.llama-services",
"health_url": "http://localhost:8000/health",
"health_timeout": 30,
"models": {
"planner": "llama-server-planner.service",
"coder": "llama-server-coder.service"
}
}Inside Claude Code:
You: list models
You: swap to planner
You: <discuss architecture, define interfaces>
You: swap to coder and implement the planThat's it. Context is preserved across swaps.
You can also generate new model configs directly:
You: create a model config named "reasoning" for /models/qwen3-30b.gguf with 8192 contextClaude Code CLI
|
| Anthropic Messages API
v
LiteLLM Proxy (:4000) <-- translates Anthropic -> OpenAI format
|
| OpenAI Chat Completions API
v
llama-server (:8000) <-- model weights swapped via service manager
^
|
mcp-llama-swap <-- this project (launchctl or systemd)Claude Code speaks Anthropic format. LiteLLM translates to OpenAI format for llama-server. This MCP server manages which model service is loaded via launchctl (macOS) or systemd (Linux).
Conversation context survives swaps because Claude Code holds the full message history client-side and re-sends it with every request.
Define aliases for your models. Only mapped models are available. Other service configs in the directory are ignored.
macOS:
{
"plists_dir": "~/.llama-plists",
"health_url": "http://localhost:8000/health",
"health_timeout": 30,
"models": {
"planner": "qwen35-35b-a3b-thinking.plist",
"coder": "qwen3-coder.plist",
"fast": "glm-4-7-flash.plist"
}
}Linux:
{
"services_dir": "~/.llama-services",
"health_url": "http://localhost:8000/health",
"health_timeout": 30,
"models": {
"planner": "llama-server-planner.service",
"coder": "llama-server-coder.service"
}
}Swap using your aliases: "swap to coder", "swap to planner".
Set "models": {} to auto-discover all service configs. Filenames (without extension) become the aliases.
macOS:
{
"plists_dir": "~/.llama-plists",
"models": {}
}Linux:
{
"services_dir": "~/.llama-services",
"models": {}
}Tool | Description |
| Lists all configured models with load status and current mode |
| Returns the alias of the currently loaded model |
| Unloads current model, loads the specified one, waits for health check |
| Generates a new launchd plist (macOS) or systemd unit (Linux) for a model |
Resource | Description |
| Current configuration as JSON |
| Current model status, health, and platform info |
Prompt | Description |
| Guided plan-then-implement workflow template |
macOS with launchctl, or Linux with systemd
llama-server (llama.cpp) installed
Model configurations as service files (launchd plists or systemd units)
Python 3.10+
Claude Code CLI pointed at a LiteLLM proxy
pip install mcp-llama-swappip install litellmCreate litellm_config.yaml:
model_list:
- model_name: "*"
litellm_params:
model: "openai/*"
api_base: "http://localhost:8000/v1"
api_key: "sk-none"
litellm_settings:
drop_params: true
request_timeout: 300Start it:
litellm --config litellm_config.yaml --port 4000On macOS, you can use the included ai.litellm.proxy.plist.template to run it as a persistent launchd service (see setup.sh).
Add to ~/.zshrc (macOS) or ~/.bashrc (Linux):
export ANTHROPIC_BASE_URL="http://localhost:4000"
export ANTHROPIC_API_KEY="sk-none"
export ANTHROPIC_MODEL="local"Add to ~/.claude.json:
{
"mcpServers": {
"llama-swap": {
"command": "uvx",
"args": ["mcp-llama-swap"],
"env": {
"LLAMA_SWAP_CONFIG": "/absolute/path/to/config.json"
}
}
}
}Copy config.example.json (macOS) or config.example.linux.json (Linux) and edit with your model aliases and service filenames.
You can create service configs manually, or use the create_model_config MCP tool inside Claude Code:
You: create a model config named "coder" for /path/to/model.gguf with 8192 contextThis generates the appropriate launchd plist (macOS) or systemd unit file (Linux) in your services directory.
If you prefer a one-shot setup on macOS, clone this repo and run:
git clone https://github.com/oussama-kh/mcp-llama-swap.git ~/mcp-llama-swap
cd ~/mcp-llama-swap
chmod +x setup.sh
./setup.shThe script creates a virtual environment, installs dependencies, configures the LiteLLM launchd service, and prints the exact config to add.
config.json fields:
Field | Default | Description |
|
| Directory containing model service configs |
| — | macOS alias for |
| — | Linux alias for |
|
| llama-server health endpoint |
|
| Seconds to wait for health check after loading |
|
| Alias-to-filename map. Empty = directory mode |
|
| Service manager: |
|
| macOS only: |
Override config path via the LLAMA_SWAP_CONFIG environment variable.
Models are managed as launchd services via plist files. Two launchctl modes are available:
Legacy (default): Uses launchctl load/unload/list. Works on all macOS versions.
Modern: Uses launchctl bootstrap/bootout/print. The officially supported API on newer macOS. Enable with "launchctl_mode": "modern" in config.
Models are managed as systemd user services. Unit files in services_dir are symlinked to ~/.config/systemd/user/ and managed via systemctl --user start/stop.
LiteLLM not translating correctly: Check /tmp/litellm.stderr.log. Verify llama-server is running: curl http://localhost:8000/health.
Model swap times out: Increase health_timeout in config.json. Large models may need 30+ seconds to load weights into memory.
Claude Code cannot find the MCP server: Verify the LLAMA_SWAP_CONFIG path is absolute. Test directly: python -m mcp_llama_swap.
Mapped model not found: The service filename in models must match an actual file in your services directory.
systemd service won't start: Check journalctl --user -u llama-server-<name> for errors. Ensure llama-server is in your PATH.
launchctl modern mode issues: If bootstrap/bootout commands fail, fall back to "launchctl_mode": "legacy" in config.
# Install with test dependencies
pip install -e ".[test]"
# Run tests
pytest -vThis project enables a two-phase AI coding workflow entirely on local hardware:
Planning phase: Load a reasoning model (e.g., Qwen3.5-35B-A3B with thinking). Discuss architecture, define interfaces, decompose requirements.
Implementation phase: Swap to a coding model (e.g., Qwen3-Coder-30B). Execute the plan file by file with full conversation context from the planning phase.
No cloud APIs. No data leaving your machine. No context loss between phases.
Apache-2.0
Generate a new service config for a llama-server model.
Creates a launchd plist (macOS) or systemd unit (Linux) that can be used with swap_model.
Args: name: Short alias for the model (e.g., "coder", "planner") model_path: Absolute path to the GGUF model file context_size: Context window size (default: 4096) gpu_layers: Number of GPU layers, -1 for all (default: -1) port: Port for llama-server (default: 8000)
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | ||
| model_path | Yes | ||
| context_size | No | ||
| gpu_layers | No | ||
| port | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It describes the output (launchd plist or systemd unit) and parameters, but does not disclose where files are saved, whether existing files are overwritten, or required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently structured: a clear summary sentence, followed by platform details, then a well-formatted argument list. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists, the description doesn't need to explain return values. It covers generation purpose, parameter roles, and integration with swap_model. Minor omission: no mention of output file location or naming convention.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, yet the description provides detailed docstring-style explanations for all 5 parameters (e.g., 'Short alias for the model', 'Absolute path to the GGUF model file'), adding significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Generate a new service config for a llama-server model' and mentions creating platform-specific service files for use with swap_model. This clearly distinguishes it from siblings like swap_model (which activates) and list_models (which queries).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It states the configs are for use with swap_model, providing contextual guidance on when to use this tool. However, it lacks explicit 'when not to use' or alternatives for cases where configs already exist.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get the currently loaded llama.cpp model.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description implies a read operation via 'Get', but does not explicitly confirm no side effects or disclose any behavioral traits. With zero parameters and a trivial operation, it is minimally adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, concise sentence that contains no extraneous information. It is well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity (0 params, output schema exists), the description is complete enough. It tells the agent exactly what the tool retrieves.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist (input schema is empty), so baseline 4 applies. The description adds no param info, but none is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get' and the specific resource 'the currently loaded llama.cpp model', distinguishing it from siblings like list_models (all models) and swap_model (change).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like list_models. While the purpose is clear, the description does not provide context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List available llama.cpp model configurations and their load status.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden; it discloses that the tool lists configurations with load status but does not elaborate on other behavioral aspects like read-only nature or potential side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no unnecessary words; every element serves the purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple listing tool with no parameters and an output schema, the description adequately covers the tool's purpose and result, though it could mention whether the operation is read-only or requires authentication.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With zero parameters and 100% schema coverage, the description need not add parameter details. Baseline of 4 is appropriate as the description adds no confusion and matches the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'List' and clearly identifies the resource as 'available llama.cpp model configurations and their load status', which distinguishes it from sibling tools like create_model_config, get_current_model, and swap_model.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when one needs to see available models and their load status, but it does not explicitly state when not to use this tool or mention alternatives beyond the sibling names.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Swap to a different llama.cpp model.
Unloads any currently loaded model, loads the requested one, and waits for the health endpoint to confirm readiness.
Args: model: Alias of the model to load
| Name | Required | Description | Default |
|---|---|---|---|
| model | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behaviors: unloads current model, loads requested, waits for health endpoint. No annotations provided, so description carries burden; it does well but could mention potential service disruption.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Very concise: three sentences plus args. Front-loaded with purpose, no fluff. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers swap process and readiness check. Doesn't mention error handling or non-existent models, but output schema exists to capture return. Fairly complete for a simple swap tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Parameter 'model' is described as 'Alias of the model to load', adding meaning beyond the schema's string type. With 0% schema coverage, this is essential and adequate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool swaps to a different llama.cpp model, unloading current and loading new, with readiness check. It distinguishes from siblings like create_model_config, get_current_model, list_models.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (loading a different model) but lacks explicit when-not or alternative guidance. No mention of prerequisites or comparison with siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v1.0.0create_model_configget_current_modellist_modelsswap_modelScored across 4 tools
Each tool has a clearly distinct function: creating configs, listing models, getting the current model, and swapping to a different model. No overlap exists.
All tool names follow a consistent verb_noun pattern in snake_case (create_model_config, get_current_model, list_models, swap_model), making them predictable.
Four tools is well-scoped for the server's purpose of managing llama.cpp model configurations and swapping. Each tool serves a necessary role without redundancy.
The set covers the key operations: creating a config, listing models, checking the current model, and swapping. However, there is no tool to delete or update a model config, which is a minor gap.
Share context and questions between Claude instances — VS Code, claude.ai web, and mobile.
Source-checked CLI guides and model-aware planning for Claude Code, Codex, and Grok Build.
Persistent context for Claude. Your AI always knows your projects and next actions across sessions.
No-data MCP handoff for local Claude Code to Codex harness moves. $49 lifetime.
