Skip to main content
Glama

llama-mcp

An MCP server that exposes a llama.cpp server (llama-server) as tools, so an MCP host such as Claude Code can hand work to a local model.

Tools

Tool

llama-server endpoint

Purpose

chat

POST /v1/chat/completions

Prompt (plus optional system prompt) → reply

complete

POST /completion

Raw text continuation, no chat template

tokenize

POST /tokenize

Text → token ids

models

GET /v1/models

Model ids and whether each is loaded

health

GET /health

ok once a model is loaded

Related MCP server: bionic-mcp

Usage

Start llama-server, then register the MCP server with Claude Code:

llama-server -m model.gguf --port 8080
claude mcp add llama -- uv run --project ~/Projects/llama-mcp llama-mcp --url http://localhost:8080

Configuration:

  • --url / LLAMA_URL: llama-server base URL (default http://localhost:8080)

  • --model / LLAMA_MODEL: model id to use when a tool call names none. Required when llama-server runs in router mode (started without -m/-hf, offering several models); chat, complete and tokenize also take a per-call model argument.

  • LLAMA_API_KEY: sent as a bearer token, for a llama-server started with --api-key

  • --timeout: request timeout in seconds (default 300)

  • --http PORT: serve streamable HTTP at http://127.0.0.1:PORT/mcp instead of stdio, for testing with MCP Inspector or claude mcp add --transport http

Development

uv sync
uv run pytest
uv run ruff check && uv run ruff format --check
uv run pyrefly check

Available Tools

5 tools
chatC

Send a prompt to the local model through its chat template and return the reply.

    Leave `temperature` unset to use llama-server's own sampling settings.
    
ParametersJSON Schema
NameRequiredDescriptionDefault
modelNo
promptYes
systemNo
max_tokensNo
temperatureNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full disclosure burden. It notes that the prompt goes through the chat template and that the reply is returned, plus one useful default about temperature, but says nothing about model-loading requirements, failure modes, timeouts, or whether an unloaded model is auto-loaded.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the core action and followed by the one non-obvious default. No filler, though the awkward indented line break slightly hurts readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be described, but for a 5-parameter generation tool with zero annotation and zero schema-description coverage, the description is too thin. Model selection, system-prompt semantics, and max_tokens limits are all left to inference.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across 5 parameters, so the description must compensate and largely does not. It clarifies temperature's default-null behavior relative to llama-server sampling, but leaves model, prompt, system, and max_tokens entirely unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a concrete verb (send a prompt), the resource (the local model via its chat template), and the outcome (return the reply). An agent understands the operation, but the sibling 'complete' is not mentioned, so the boundary between chat-template generation and plain completion is left implicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to prefer this over 'complete', 'tokenize', or 'models', nor any prerequisites (e.g., whether a model must be loaded first). The only conditional note is about the temperature parameter, which is a parameter hint rather than usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

completeB

Continue raw text with the local model, with no chat template applied.

    Leave `temperature` unset to use llama-server's own sampling settings.
    
ParametersJSON Schema
NameRequiredDescriptionDefault
modelNo
promptYes
n_predictNo
temperatureNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It usefully discloses one behavior — that leaving `temperature` unset delegates sampling to llama-server — but says nothing about resource cost, latency, whether the call is stateful, or how the model is selected. Partial disclosure for a no-annotation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short, front-loaded sentences with no filler; the key differentiator (no chat template) comes first. Minor stray indentation/whitespace in the text is the only blemish.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be described, and the tool is conceptually simple. But for a 4-parameter tool with zero annotation and zero schema-description coverage, the description is too thin to fully orient an agent on parameter behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% across 4 parameters, so the description must compensate, yet it only clarifies `temperature`. The meaning of `model`, `prompt`, and `n_predict` (including its 256 default and relationship to output length) is left entirely unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Continue raw text with the local model') and adds the distinguishing constraint 'with no chat template applied', which implicitly separates it from the `chat` sibling. It never names the sibling, so differentiation is inferred rather than explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'no chat template applied' implies this is the tool for raw/completion-style use as opposed to `chat`, but there is no explicit when-to-use statement, no exclusion, and no named alternative. Usage is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

healthA

Report whether llama-server is up and has a model loaded.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry behavioral disclosure. It does state the two conditions it checks (server up, model loaded), implying a read-only probe, but says nothing about failure behavior, latency, or what an unhealthy response looks like beyond what the output schema presumably covers.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler; every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter health probe with an output schema present, the description covers the essential purpose adequately. Return-value details are rightly left to the output schema, though a note on failure modes would complete it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing for the description to disambiguate; the baseline for a parameterless tool applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Report') and resource ('llama-server' status plus model-loaded state), which is a distinct diagnostic function from the sibling tools (chat, complete, tokenize, models). It's clear what the tool does, though it doesn't explicitly contrast itself with siblings like 'models', which could also surface model information.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the name and nature of a health check, but the description offers no explicit when-to-use or when-not-to-use guidance, nor any mention of alternatives for verifying server/model availability.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

modelsA

List the models llama-server offers, with their status.

    A single-model llama-server always reports `loaded`. In router mode a model is `unloaded` until
    a request names it, and the first such request waits while it loads.
    
ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral burden and does disclose meaningful traits: single-model servers always report loaded, while router-mode models are unloaded until a request names them and the first such request waits for loading. It does not mention read-only safety or idempotency, but the listing nature is implicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two tightly scoped parts: a front-loaded purpose sentence followed by essential status semantics. Every sentence earns its place with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple listing purpose, an empty input schema, and an existing output schema, the description covers what the tool does and how to interpret model statuses. The only notable gap is the absence of explicit usage guidance relative to sibling tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so per the scoring rules the baseline is 4. There is no parameter information to clarify or omit.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (List) and resource (the models llama-server offers) with the added qualifier 'with their status'. This is clearly distinct from sibling tools like chat, complete, tokenize, and health, though it does not explicitly name or contrast them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains what statuses mean but never says when to call this tool versus alternatives, nor does it describe prerequisites or follow-up actions. There is no explicit when-to-use or when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tokenizeC

Return the model's token ids for text.

ParametersJSON Schema
NameRequiredDescriptionDefault
textYes
modelNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden, and it discloses only the return value. It says nothing about the optional `model` parameter, what model is used by default, whether authentication or rate limits apply, or any other operational trait.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with zero filler; nothing could be removed without losing content. It is appropriately sized for the operation, though brevity here reflects under-specification elsewhere rather than a virtue of completeness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained, but for a 2-param tool with no annotations and 0% schema description coverage the description is too thin. The behavior of the optional `model` argument and any usage context are entirely absent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, yet it only names `text` implicitly and gestures at 'the model'. It never explains that `model` is optional/defaults to null, so half the parameter surface remains undocumented in both schema and description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Return') and resource ('the model's token ids for `text`'), which unambiguously distinguishes it from the sibling tools chat/complete/models/health, none of which tokenize. It is clear but terse, offering no scope detail beyond the single operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives, no mention of prerequisites, and no exclusions. The text implies inspection/measurement usage but leaves the agent to infer all context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 5 tool updatesv0.1.0
    • First observedchat
    • First observedcomplete
    • First observedhealth
    • First observedmodels
    • First observedtokenize

TDQS

B3.2/5.0

Scored across 5 tools

Disambiguation4/5

chat and complete both send text to the model and return output, so their boundary rests entirely on the chat-template distinction, which the descriptions do clarify. tokenize, models, and health are each clearly distinct from that pair and from each other.

Naming Consistency3/5

All names are single lowercase words in a consistent casing style, which keeps them readable. However, the set mixes bare verbs (chat, complete, tokenize) with bare nouns (models, health) with no predictable verb_noun pattern.

Tool Count5/5

Five tools is well-scoped for a thin wrapper over a local llama-server: two generation modes, tokenizer access, model listing, and a health probe. Each tool earns its place with no obvious redundancy.

Completeness3/5

Core inference and diagnostics are covered, but common llama-server capabilities are absent: embeddings, explicit model selection/load-unload for router mode, and detokenization. Agents can still work around these gaps but will hit dead ends for embedding workflows.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • F
    license
    A
    quality
    D
    maintenance
    MCP server that connects LLM agents to a local LM Studio instance, enabling model management, OpenAI-compatible chat completions, text completions, and embeddings through a set of tools.
    9
    1
    -
  • A
    license
    Not graded
    quality
    B
    maintenance
    Bridges LM Studio local models to Claude Code as MCP tools, enabling private and free AI-assisted subtasks. Includes respond and models tools with automatic model selection and structured output.
    MIT