Skip to main content
Glama
ahays248

llama-mcp-server

by ahays248
OverviewSchemaRelated ServersScoreDiscussions
TypeScript
Local

llama-mcp-server

MCP server bridging Claude Code to local llama.cpp. Run local LLMs alongside Claude for experimentation, testing, and cost-effective inference.

Requirements

  • Node.js 18+

  • llama.cpp with llama-server built

  • A GGUF model file

Related MCP server: agent-skill-loader

Installation

npm install llama-mcp-server

Or clone and build from source:

git clone https://github.com/ahays248/llama-mcp-server
cd llama-mcp-server
npm install
npm run build

Configuration

Configure via environment variables:

Variable

Description

Default

LLAMA_SERVER_URL

URL of llama-server

http://localhost:8080

LLAMA_SERVER_TIMEOUT

Request timeout in ms

30000

LLAMA_MODEL_PATH

Path to GGUF model file

(none)

LLAMA_SERVER_PATH

Path to llama-server binary

llama-server

Usage with Claude Code

Option 1: Plugin Installation (Recommended)

Due to a known bug in Claude Code, non-plugin MCP servers may connect but not expose their tools. The workaround is to install llama-mcp-server as a plugin via a local marketplace.

Step 1: Create the marketplace structure

llama-marketplace/
├── .claude-plugin/
│   └── marketplace.json
└── plugins/
    └── llama/
        ├── .claude-plugin/
        │   └── plugin.json
        └── .mcp.json

Step 2: Create marketplace.json

// llama-marketplace/.claude-plugin/marketplace.json
{
  "name": "llama-marketplace",
  "description": "Local marketplace for llama.cpp MCP plugin",
  "owner": {
    "name": "Your Name"
  },
  "plugins": [
    {
      "name": "llama",
      "description": "llama.cpp MCP server for local LLM inference",
      "source": "./plugins/llama"
    }
  ]
}

Step 3: Create plugin.json

// llama-marketplace/plugins/llama/.claude-plugin/plugin.json
{
  "name": "llama",
  "version": "0.1.0",
  "description": "llama.cpp MCP server for local LLM inference"
}

Step 4: Create .mcp.json

// llama-marketplace/plugins/llama/.mcp.json
{
  "mcpServers": {
    "llama": {
      "command": "npx",
      "args": ["-y", "llama-mcp-server"],
      "env": {
        "LLAMA_SERVER_URL": "http://localhost:8080",
        "LLAMA_MODEL_PATH": "/path/to/your/model.gguf",
        "LLAMA_SERVER_PATH": "/path/to/llama-server"
      }
    }
  }
}

Step 5: Install the plugin

# Add the local marketplace
claude plugin marketplace add /path/to/llama-marketplace

# Install the plugin
claude plugin install llama@llama-marketplace

# Restart Claude Code

After restart, tools will appear as mcp__plugin_llama_llama__*.

Option 2: Direct MCP Configuration

Note: This method may not work due to the bug mentioned above. If tools don't appear after adding the server, use Option 1.

Add to your Claude Code MCP configuration:

claude mcp add llama -e LLAMA_SERVER_URL=http://localhost:8080 -e LLAMA_MODEL_PATH=/path/to/model.gguf -e LLAMA_SERVER_PATH=/path/to/llama-server -- npx -y llama-mcp-server

Or add manually to ~/.claude.json:

{
  "mcpServers": {
    "llama": {
      "command": "npx",
      "args": ["-y", "llama-mcp-server"],
      "env": {
        "LLAMA_SERVER_URL": "http://localhost:8080",
        "LLAMA_MODEL_PATH": "/path/to/your/model.gguf",
        "LLAMA_SERVER_PATH": "/path/to/llama-server"
      }
    }
  }
}

Tools

Server Tools

Tool

Description

llama_health

Check if llama-server is running and get status

llama_props

Get or set server properties

llama_models

List available/loaded models

llama_slots

View current slot processing state

llama_metrics

Get Prometheus-compatible metrics

Token Tools

Tool

Description

llama_tokenize

Convert text to token IDs

llama_detokenize

Convert token IDs back to text

llama_apply_template

Format chat messages using model's template

Inference Tools

Tool

Description

llama_complete

Generate text completion from a prompt

llama_chat

Chat completion (OpenAI-compatible)

llama_embed

Generate embeddings for text

llama_infill

Code completion with prefix and suffix context

llama_rerank

Rerank documents by relevance to a query

Model Management Tools

Tool

Description

llama_load_model

Load a model (router mode only)

llama_unload_model

Unload the current model (router mode only)

LoRA Tools

Tool

Description

llama_lora_list

List loaded LoRA adapters

llama_lora_set

Set LoRA adapter scales

Process Control Tools

Tool

Description

llama_start

Start llama-server as a child process

llama_stop

Stop the llama-server process

Example: Starting llama-server and Running Inference

User: Start llama-server with my local model

Claude: I'll start llama-server for you.
[Uses llama_start tool with model path]

User: Generate a haiku about coding

Claude: Let me use the local model for that.
[Uses llama_complete tool]

Result:
Lines of code cascade
Through the silent morning hours
Bugs flee from the light

Development

# Run tests
npm test

# Type check
npm run typecheck

# Build
npm run build

# Watch mode for development
npm run dev

Troubleshooting

Tools don't appear in Claude Code

Symptom: Server shows "Connected" in claude mcp list but no llama_* tools are available.

Cause: Known bug in Claude Code where non-plugin MCP servers don't expose tools (#12164).

Solution: Use the plugin installation method (Option 1 above).

HTTP 501 errors for certain tools

Some tools require specific server configurations:

Tool

Requirement

llama_metrics

Start llama-server with --metrics flag

llama_embed

Start llama-server with --embedding flag or use an embedding model

llama_infill

Use a model with fill-in-middle support (e.g., CodeLlama, DeepSeek Coder)

llama_rerank

Use a reranker model

llama_load_model / llama_unload_model

llama-server must be in router mode

Connection refused errors

Symptom: Cannot connect to llama-server at http://localhost:8080

Solutions:

  1. Use llama_start to start the server, or

  2. Start llama-server manually: llama-server -m /path/to/model.gguf

  3. Check that LLAMA_SERVER_URL matches where llama-server is running

WSL/Windows path issues

When running in WSL, ensure paths use Linux format:

  • ✓ /home/user/models/model.gguf

  • ✗ C:\Users\user\models\model.gguf

License

MIT

Install Server
A
license - permissive license
A
quality
D
maintenance

How are these scores calculated?

Maintenance

–Maintainers
–Response time
–Release cycle
–Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Tools

View all tools

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/ahays248/llama-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server