llama-mcp
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@llama-mcpask the local llama model to explain recursion in one sentence"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
llama-mcp
An MCP server that exposes a llama.cpp server (llama-server) as tools, so an MCP host such as Claude Code can hand work to a local model.
Tools
Tool | llama-server endpoint | Purpose |
|
| Prompt (plus optional system prompt) → reply |
|
| Raw text continuation, no chat template |
|
| Text → token ids |
|
| Model ids and whether each is loaded |
|
|
|
Related MCP server: bionic-mcp
Usage
Start llama-server, then register the MCP server with Claude Code:
llama-server -m model.gguf --port 8080
claude mcp add llama -- uv run --project ~/Projects/llama-mcp llama-mcp --url http://localhost:8080Configuration:
--url/LLAMA_URL: llama-server base URL (defaulthttp://localhost:8080)--model/LLAMA_MODEL: model id to use when a tool call names none. Required when llama-server runs in router mode (started without-m/-hf, offering several models);chat,completeandtokenizealso take a per-callmodelargument.LLAMA_API_KEY: sent as a bearer token, for a llama-server started with--api-key--timeout: request timeout in seconds (default 300)--http PORT: serve streamable HTTP athttp://127.0.0.1:PORT/mcpinstead of stdio, for testing with MCP Inspector orclaude mcp add --transport http
Development
uv sync
uv run pytest
uv run ruff check && uv run ruff format --check
uv run pyrefly checkAvailable Tools
5 toolschatC
Send a prompt to the local model through its chat template and return the reply.
Leave `temperature` unset to use llama-server's own sampling settings.
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | ||
| prompt | Yes | ||
| system | No | ||
| max_tokens | No | ||
| temperature | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full disclosure burden. It notes that the prompt goes through the chat template and that the reply is returned, plus one useful default about temperature, but says nothing about model-loading requirements, failure modes, timeouts, or whether an unloaded model is auto-loaded.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the core action and followed by the one non-obvious default. No filler, though the awkward indented line break slightly hurts readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be described, but for a 5-parameter generation tool with zero annotation and zero schema-description coverage, the description is too thin. Model selection, system-prompt semantics, and max_tokens limits are all left to inference.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across 5 parameters, so the description must compensate and largely does not. It clarifies temperature's default-null behavior relative to llama-server sampling, but leaves model, prompt, system, and max_tokens entirely unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a concrete verb (send a prompt), the resource (the local model via its chat template), and the outcome (return the reply). An agent understands the operation, but the sibling 'complete' is not mentioned, so the boundary between chat-template generation and plain completion is left implicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to prefer this over 'complete', 'tokenize', or 'models', nor any prerequisites (e.g., whether a model must be loaded first). The only conditional note is about the temperature parameter, which is a parameter hint rather than usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
completeB
Continue raw text with the local model, with no chat template applied.
Leave `temperature` unset to use llama-server's own sampling settings.
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | ||
| prompt | Yes | ||
| n_predict | No | ||
| temperature | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It usefully discloses one behavior — that leaving `temperature` unset delegates sampling to llama-server — but says nothing about resource cost, latency, whether the call is stateful, or how the model is selected. Partial disclosure for a no-annotation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short, front-loaded sentences with no filler; the key differentiator (no chat template) comes first. Minor stray indentation/whitespace in the text is the only blemish.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be described, and the tool is conceptually simple. But for a 4-parameter tool with zero annotation and zero schema-description coverage, the description is too thin to fully orient an agent on parameter behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% across 4 parameters, so the description must compensate, yet it only clarifies `temperature`. The meaning of `model`, `prompt`, and `n_predict` (including its 256 default and relationship to output length) is left entirely unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Continue raw text with the local model') and adds the distinguishing constraint 'with no chat template applied', which implicitly separates it from the `chat` sibling. It never names the sibling, so differentiation is inferred rather than explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'no chat template applied' implies this is the tool for raw/completion-style use as opposed to `chat`, but there is no explicit when-to-use statement, no exclusion, and no named alternative. Usage is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
healthA
Report whether llama-server is up and has a model loaded.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry behavioral disclosure. It does state the two conditions it checks (server up, model loaded), implying a read-only probe, but says nothing about failure behavior, latency, or what an unhealthy response looks like beyond what the output schema presumably covers.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler; every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter health probe with an output schema present, the description covers the essential purpose adequately. Return-value details are rightly left to the output schema, though a note on failure modes would complete it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is nothing for the description to disambiguate; the baseline for a parameterless tool applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Report') and resource ('llama-server' status plus model-loaded state), which is a distinct diagnostic function from the sibling tools (chat, complete, tokenize, models). It's clear what the tool does, though it doesn't explicitly contrast itself with siblings like 'models', which could also surface model information.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the name and nature of a health check, but the description offers no explicit when-to-use or when-not-to-use guidance, nor any mention of alternatives for verifying server/model availability.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
modelsA
List the models llama-server offers, with their status.
A single-model llama-server always reports `loaded`. In router mode a model is `unloaded` until
a request names it, and the first such request waits while it loads.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full behavioral burden and does disclose meaningful traits: single-model servers always report loaded, while router-mode models are unloaded until a request names them and the first such request waits for loading. It does not mention read-only safety or idempotency, but the listing nature is implicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two tightly scoped parts: a front-loaded purpose sentence followed by essential status semantics. Every sentence earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple listing purpose, an empty input schema, and an existing output schema, the description covers what the tool does and how to interpret model statuses. The only notable gap is the absence of explicit usage guidance relative to sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so per the scoring rules the baseline is 4. There is no parameter information to clarify or omit.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (List) and resource (the models llama-server offers) with the added qualifier 'with their status'. This is clearly distinct from sibling tools like chat, complete, tokenize, and health, though it does not explicitly name or contrast them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains what statuses mean but never says when to call this tool versus alternatives, nor does it describe prerequisites or follow-up actions. There is no explicit when-to-use or when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tokenizeC
Return the model's token ids for text.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | ||
| model | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden, and it discloses only the return value. It says nothing about the optional `model` parameter, what model is used by default, whether authentication or rate limits apply, or any other operational trait.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with zero filler; nothing could be removed without losing content. It is appropriately sized for the operation, though brevity here reflects under-specification elsewhere rather than a virtue of completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained, but for a 2-param tool with no annotations and 0% schema description coverage the description is too thin. The behavior of the optional `model` argument and any usage context are entirely absent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate, yet it only names `text` implicitly and gestures at 'the model'. It never explains that `model` is optional/defaults to null, so half the parameter surface remains undocumented in both schema and description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Return') and resource ('the model's token ids for `text`'), which unambiguously distinguishes it from the sibling tools chat/complete/models/health, none of which tokenize. It is clear but terse, offering no scope detail beyond the single operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives, no mention of prerequisites, and no exclusions. The text implies inspection/measurement usage but leaves the agent to infer all context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
5 tool updates
v0.1.0- First observed
chat - First observed
complete - First observed
health - First observed
models - First observed
tokenize
TDQS
Scored across 5 tools
chat and complete both send text to the model and return output, so their boundary rests entirely on the chat-template distinction, which the descriptions do clarify. tokenize, models, and health are each clearly distinct from that pair and from each other.
All names are single lowercase words in a consistent casing style, which keeps them readable. However, the set mixes bare verbs (chat, complete, tokenize) with bare nouns (models, health) with no predictable verb_noun pattern.
Five tools is well-scoped for a thin wrapper over a local llama-server: two generation modes, tokenizer access, model listing, and a health probe. Each tool earns its place with no obvious redundancy.
Core inference and diagnostics are covered, but common llama-server capabilities are absent: embeddings, explicit model selection/load-unload for router mode, and detokenization. Agents can still work around these gaps but will hit dead ends for embedding workflows.
Maintenance
Related MCP Connectors
Nifty's MCP server — exposes tasks, projects, messages, and files as tools for AI agents.
MCP server for your apps' tools and custom tools, plus hosted AI agents and approval-gated workflows
A comprehensive Model Context Protocol (MCP) server that enables AI assistants to interact with yo…
AI Reasoning Cache & Consensus Layer with 11 MCP tools via Streamable HTTP.
Related MCP Servers
- FlicenseAqualityDmaintenanceMCP server that connects LLM agents to a local LM Studio instance, enabling model management, OpenAI-compatible chat completions, text completions, and embeddings through a set of tools.91-
- AlicenseNot gradedqualityBmaintenanceBridges LM Studio local models to Claude Code as MCP tools, enabling private and free AI-assisted subtasks. Includes respond and models tools with automatic model selection and structured output.MIT
- AlicenseNot gradedqualityCmaintenanceExposes local LM Studio language models as MCP tools, enabling chat completions and model listing through a local OpenAI-compatible API without requiring API keys.MIT

LocalAiMCPofficial
AlicenseNot gradedqualityBmaintenanceEnables natural-language control of a LocalAI server, exposing its full API as MCP tools for model management, inference, monitoring, media processing, and advanced system operations.MIT