Local AI MCP Servers
Allows fetching models directly from Hugging Face into the local model machine through the fetch_model tool with source="huggingface".
Provides tools for managing an Ollama model machine (pulling/removing models, building custom Ollama variants, running LoRA training, switching the inference service) and for querying local Ollama models with schema-validated JSON output.
Local-AI MCP servers: Ollama & vLLM model management and local LLM access for Claude, ChatGPT and other MCP clients
Two self-hostable Model Context Protocol (MCP) servers for running and steering your own local language models: manage an Ollama/vLLM model machine, and query a local LLM (Ollama, vLLM or an OpenAI-compatible backend) for language work with schema-validated JSON output. Self-hosted and privacy-first: the models and their machine stay on your own hardware, and neither server process reaches out to a public address on its own. Version 1.0.0, single-operator tool, actively used by the author.
Both servers speak the Model Context Protocol (MCP), so an AI tool such as Claude
or ChatGPT can use them. Each server's own traffic goes only to your own computer
or your local network; a start-up guard rejects public addresses. Installing the
two direct dependencies needs the internet once, at setup time. Fetching a model
is a separate matter: in mcp-modelmanager, calling fetch_model is a
deliberate, tool-triggered connection from your model machine to a public
model registry, at whatever point you invoke it, not something limited to
setup time. The default source="ollama" makes the machine's own Ollama
daemon pull from registry.ollama.ai for plain model names, or from whatever
other registry a prefixed name such as hf.co/user/repo points at;
source="huggingface" instead reaches Hugging Face directly. switch_service
never reaches either registry itself;
it refuses and asks for fetch_model first if the model is not already on the
machine. Each server stands on its own: pick one, or run both.
A caveat: the AI tool you drive these servers with (Claude, ChatGPT, etc.) is itself a hosted service in the usual case. The instructions you give it and the tool inputs and results it handles pass through that provider, exactly as any other MCP call would. "Local" here means the models, the model machine and its data; it is not a claim that nothing you type ever reaches a third party. Drive them with a local model if that matters to you.
The two servers
mcp-modelmanager -> mcp-modelmanager/
Manage your own model machine through an AI tool: read state (GPU memory, disk, running service, containers), pull and remove models, switch the vLLM inference service to a different model, build custom Ollama variants and run LoRA training. There is deliberately no tool for free remote control: every operation is a fixed template and every parameter first passes an allowlist, checked independently on both sides. This server ships both halves: the client/server code the AI tool talks to, and the machine-side scripts, wrapper and network-fence templates that run on and lock down the model machine.
mcp-ollama-vllm -> mcp-ollama-vllm/
Call a local model directly from an AI session for the language work: writing, summarizing, classifying, extracting structured data. Its distinctive feature is schema-validated JSON output, checked by its own dependency-free validator and retried automatically on a violation, the same way whether the configured backend is Ollama or vLLM. It is a model call, not an agent: no tools, no file access, no shell.
The two are complementary. mcp-ollama-vllm asks a model (read-only, no
management rights); mcp-modelmanager manages the machine. Keeping them
separate means a query call never accidentally holds management rights.
Related MCP server: Ollama MCP Domain Expert Delegation
Repository layout
.
|-- README.md this file
|-- CHANGELOG.md version history (Keep a Changelog style)
|-- CONTRIBUTING.md how to contribute
|-- LICENSE MIT, covers the repo as a whole
|-- .gitignore
|-- .github/
| `-- workflows/
| `-- ci.yml lockfile check, locked-tree tests, shell lint, advisory pip-audit
|-- mcp-modelmanager/ manage your own model machine (server + machine side)
| |-- README.md server and client guide (start here)
| |-- SETUP.md client-side step-by-step setup
| |-- AGENT-SETUP.md setup procedure written for an AI agent
| |-- MACHINE.md machine-side overview (start here for the machine)
| |-- SETUP-MACHINE.md gapless machine provisioning
| |-- SECURITY.md the three-layer security model
| |-- LICENSE
| |-- .gitignore SSH-key and build patterns (kept from the server)
| |-- requirements.txt
| |-- pyproject.toml
| |-- uv.lock full resolved dependency tree, pinned
| |-- src/mcp_modelmanager/ server package (server code and shared modules)
| |-- setup/ machine-side setup scripts
| |-- vm_side/ machine-side wrapper and job runners
| `-- security/ network-fence templates (sshd, firewall, WireGuard)
`-- mcp-ollama-vllm/ call local models with schema-checked JSON
|-- README.md server guide (start here)
|-- AGENT-SETUP.md setup procedure written for an AI agent
|-- SECURITY.md what the bridge does and does not protect
|-- LICENSE
|-- requirements.txt
|-- pyproject.toml
|-- uv.lock full resolved dependency tree, pinned
`-- src/mcp_ollama_vllm/ server package (server code)Quickstart
Each server has its own guide; start with the README in its folder.
mcp-modelmanager: read mcp-modelmanager/README.md and follow mcp-modelmanager/SETUP.md for the client side. For the model machine itself, start from mcp-modelmanager/MACHINE.md.mcp-ollama-vllm: read mcp-ollama-vllm/README.md; it has no separate SETUP.md (the README covers setup, there is no machine side and no management rights to document separately), and its SECURITY.md states the boundaries of the bridge.
mcp-modelmanager needs Python 3.11 or newer, mcp-ollama-vllm 3.10 or newer;
each has only two direct, pinned dependencies (mcp and httpx); mcp
itself pulls in a number of further packages transitively (anyio, pydantic,
starlette, uvicorn, jsonschema, pyjwt among them). The full resolved
tree, direct and transitive, is pinned per server in its uv.lock; CI checks
with uv lock --check that the lock still matches pyproject.toml and runs
the tests against the exact locked tree, so a drifted lock fails CI. No
account, no paid service, no cloud is required.
Installation / Usage
Install whichever server(s) you need from PyPI:
pip install mcp-modelmanager
pip install mcp-ollama-vllmEach ships a console entry point (mcp-modelmanager / mcp-ollama-vllm) after
install, so an MCP client can point command straight at it, for example:
{
"mcpServers": {
"modelmanager": {
"command": "/path/to/.venv/bin/mcp-modelmanager",
"env": {
"MM_ACCESS": "direct",
"MM_VM_HOST": "<your-machine-or-127.0.0.1>",
"MM_VM_USER": "<your-ssh-user>",
"MM_CONTAINER_ROOT": "/srv/models"
}
},
"local-models": {
"command": "/path/to/.venv/bin/mcp-ollama-vllm",
"env": {
"LOCAL_BACKEND": "ollama",
"LOCAL_HOST": "http://localhost:11434"
}
}
}
}The required environment variables differ per server (mcp-modelmanager needs
the three MM_* fields above to start at all; mcp-ollama-vllm runs with
defaults for a local Ollama). This is only the shape; do not copy the values
verbatim. For the full, current install/registration steps, including the
checkout-based (non-PyPI) path and running from source, see each server's own
README: mcp-modelmanager/README.md
and mcp-ollama-vllm/README.md.
Security and scope
Local and self-hosted. Both server processes run on your own hardware and talk only to your own computer or your local/private network. Public addresses are rejected at start-up, and names are resolved so the guard cannot be bypassed via DNS. This governs the servers' own traffic; the AI tool that drives them is a separate service (see the caveat above), and so is a model download you explicitly trigger through
mcp-modelmanager(fetch_model): with the defaultsource="ollama"it makes the machine's own Ollama daemon pull fromregistry.ollama.ai(or from the registry a prefixed model name such ashf.co/user/repopoints at), and withsource="huggingface"it reaches Hugging Face directly, both from the model machine at the time you ask for it.switch_servicedoes not reach either registry; it refuses a model that is not already on the machine and points you atfetch_modelinstead.No credentials in this repository. No keys, tokens or passwords are stored here; SSH keys for
mcp-modelmanagerlive outside the repo in your~/.ssh, and the.gitignoreadditionally excludes key and.envpatterns.Placeholders, not real values. Every machine-specific value in the docs and templates is a
<placeholder>or an environment variable you fill in on your own machine at setup time.Each server has its own
SECURITY.md(mcp-modelmanager, mcp-ollama-vllm) that goes into the actual trust boundaries and layers, and states plainly what remains unaddressed.
License
MIT for both servers. See the LICENSE file in each server folder.
Copyright (c) 2026 Siegfried Emil Timothy Heerwagen.
Available Tools
4 toolslist_modelsA
Lists the models available on the local instance.
Returns per model the name, parameter size, quantization and on-disk size, as far as the backend reports them. With vLLM it additionally shows whether an entry is a LoRA adapter and which base model it belongs to. Sensible before any other tool, to pick a fitting and actually present model name.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It adds useful behavioral detail: results depend on backend reporting ('as far as the backend reports them'), and vLLM-specific behavior is disclosed (showing LoRA adapters and base models). This provides meaningful transparency beyond the simple operation name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the core purpose, followed by return-value details and usage timing. No wasted words, and every sentence contributes meaningful information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, parameterless, and has an output schema, so the description need not re-explain return values. It covers what the tool lists, backend-specific behavior, caveats, and when to use it, which is complete for an agent to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and the schema is empty, so there is nothing to document. The baseline of 4 applies because no parameter semantics are needed; the description is not lacking anything in this area.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Lists') and resource ('models available on the local instance'), and goes beyond a simple label by enumerating the included fields: name, parameter size, quantization, and on-disk size. This clearly distinguishes it from the sibling tools, which are about asking, structuring, and embedding.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit guidance: 'Sensible before any other tool, to pick a fitting and actually present model name.' This tells the agent when to call it, although it does not explicitly mention when-not-to-use or alternative tools, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
local_askA
Asks a local model a question and returns the answer as text.
For free-text tasks: writing, rewriting, summarizing, explaining. If a
machine-processable result is needed, use 'local_structured' instead.
Args:
model: Model name as in 'list_models', for example 'llama3.2:3b'.
prompt: The actual instruction for the model.
system: Optional role/behavior instruction.
temperature: 0 means as deterministic as possible (default), higher means more creative.
max_tokens: Optional cap on the number of generated tokens.
context: Optional text prepended to the prompt (source, excerpt, data).
think: Enables the model's reasoning trace. Off by default, because the
reasoning trace otherwise consumes the answer's token budget (with a
tight 'max_tokens' the answer then comes back empty). Only for models
with the 'thinking' capability.
| Name | Required | Description | Default |
|---|---|---|---|
| model | Yes | ||
| think | No | ||
| prompt | Yes | ||
| system | No | ||
| context | No | ||
| max_tokens | No | ||
| temperature | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and does so thoroughly. It discloses the token-budget behavior of the think parameter, warns that a tight max_tokens can cause an empty answer, and notes that thinking is only available for models with that capability. This is valuable operational context beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose, followed by concise usage guidance and a well-structured Args list. Every sentence adds meaningful information; there is no filler or redundancy. The length is justified by the number of parameters and the critical think-related warning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete for a 7-parameter tool with no annotations. It explains the return type, all parameters, usage boundaries, and a non-obvious behavioral trap. Combined with the presence of an output schema, nothing essential is missing for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description fully compensates by explaining every parameter: model references list_models, prompt is the actual instruction, system is role/behavior, temperature is determinism vs creativity, max_tokens is a cap, context is prepended text, and think explains the reasoning trace with its trade-off. This is exemplary parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Asks a local model a question and returns the answer as text.' It also distinguishes itself from the sibling local_structured by explicitly saying to use that tool for machine-processable results. The purpose is clear, specific, and differentiated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly gives usage context: 'For free-text tasks: writing, rewriting, summarizing, explaining.' It also states the alternative: 'If a machine-processable result is needed, use local_structured instead.' This gives the agent clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
local_embedA
Computes embedding vectors for a list of texts.
Useful for similarity comparisons, duplicate detection or a rough sort by
topic, without troubling a large language model.
Args:
texts: List of texts to embed.
model: Embedding model, with Ollama preset to 'nomic-embed-text'.
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | nomic-embed-text | |
| texts | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral disclosure burden. It states the core behavior and the Ollama preset for the model, but it does not mention input limits, handling of empty/large texts, or error behavior. Core behavior is clear, but some operational details are absent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded: purpose first, then use cases, then argument details. Every sentence adds useful information, and the Args section is minimal and scannable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a relatively simple embedding tool, the description covers what the tool does, when to use it, and both arguments. The output schema exists, so return values do not need to be documented here. It does not explicitly say the computation is local, but the tool name and 'Ollama preset' strongly imply it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no property descriptions, so the description must compensate. It explains that 'texts' are the texts to embed and that 'model' is the embedding model, notably adding the non-obvious Ollama preset context beyond the schema default. It could specify allowed model choices, but it covers both parameters adequately for a simple two-argument tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Computes embedding vectors for a list of texts.' It also distinguishes the tool from likely language-model siblings by noting it works 'without troubling a large language model,' so an agent can tell local_embed apart from local_ask and local_structured.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives clear use cases: similarity comparisons, duplicate detection, and rough topic sorting. It also implies when not to use a generative LLM, which is helpful for choosing between this tool and sibling tools, though it does not explicitly name alternatives or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
local_structuredA
Has a local model return a result that conforms to a JSON schema.
Uses the respective backend's schema enforcement (Ollama's 'format' field,
vLLM's 'response_format' with 'json_schema') and then additionally validates
the answer against the schema itself. If the output does not satisfy the
schema, it retries up to twice, passing the model the concrete violations.
Only then an error, but then with the invalid raw output, so it is visible
what went wrong.
Args:
model: Model name, for example 'llama3.2:3b'.
prompt: Instruction on what should be extracted from which text.
schema: JSON schema of the desired result (object with 'type', 'properties', ...).
system: Optional role/behavior instruction.
think: Enables the model's reasoning trace. Off by default, because the
reasoning trace otherwise consumes the answer's token budget. Only
for models with the 'thinking' capability.
| Name | Required | Description | Default |
|---|---|---|---|
| model | Yes | ||
| think | No | ||
| prompt | Yes | ||
| schema | Yes | ||
| system | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full behavioral disclosure. It explains validation against the schema, a retry mechanism up to twice with concrete violations, the final error containing the raw invalid output, and the token-budget tradeoff of setting think=true. This is unusually transparent and goes far beyond a minimal description.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and dense: a one-sentence purpose, a concise mechanics paragraph, and a labeled Args section. Every sentence adds operational value, and the most important information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity—multi-backend enforcement, retries, error behavior, and optional reasoning trace—the description covers nearly everything needed for correct invocation. Minor gaps: it doesn't state what happens when think=true is used on a model without the thinking capability, and it offers no explicit sibling routing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description is the sole source of parameter meaning. It defines every argument: model with an example, prompt's purpose, schema's expected structure, system as optional, and think with default, rationale, and model-capability constraint. This fully compensates for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear, specific statement: it makes a local model return a result conforming to a JSON schema. It further details the backend mechanisms (Ollama 'format', vLLM 'response_format'), which clarifies what the tool does and distinguishes it from the sibling tools local_ask and local_embed.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Intended usage is strongly implied—you use this tool when you need schema-enforced JSON output from a local model—but there is no explicit guidance about when not to use it or which sibling to prefer (e.g., local_ask for free-form text). The agent must infer tool selection from context rather than being told.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
4 tool updates
v1.0.2- First observed
list_models - First observed
local_ask - First observed
local_embed - First observed
local_structured
TDQS
Each tool targets a clearly distinct capability: listing models, free-text generation, structured JSON generation, and embeddings. local_ask and local_structured are explicitly differentiated by output type, so an agent should not confuse them.
The local_* prefix gives most tools a consistent namespace, and local_ask/local_structured/local_embed are readable. list_models breaks the pattern slightly by using verb_noun without the prefix, but this is minor and still predictable.
Four tools is well-scoped for a local AI inference server: discovery, text generation, structured generation, and embeddings cover the core capabilities without bloat or redundancy.
The set covers the essential workflows for local model interaction: find available models, ask free-text questions, get schema-validated structured answers, and compute embeddings. There are no obvious dead ends or missing core operations for the stated purpose.
Maintenance
Related MCP Connectors
- UnifAPIOAuthcom.unifapi
Hosted MCP server for live public-data APIs and Skills for AI agents.
Nifty's MCP server — exposes tasks, projects, messages, and files as tools for AI agents.
MCP server unifying ERPs, CRMs, APIs and knowledge base for Claude, ChatGPT and Gemini.
Remote MCP server for supportsheep: run AI interviews and manage support content for your blog.
Related MCP Servers
AlicenseAqualityAmaintenanceUnified MCP server for managing local model runtimes (Ollama, LM Studio, etc.), enabling provider-agnostic discovery, lifecycle management, hardware-fit checks, and delegated inference.1640Creative Commons Attribution Non Commercial No Derivatives 4.0 International- FlicenseNot gradedqualityCmaintenanceAn MCP server that lets AI agents delegate domain-specific tasks to local Ollama models, using purpose-built specialists for structured tasks like config generation, parsing, and validation.-
- AlicenseBqualityBmaintenanceAn MCP server that turns your machine or LAN of ollama nodes into a local token generator, enabling coding agents to delegate bounded processing tasks to local models and save cloud credits.51MIT
- FlicenseAqualityCmaintenanceMCP server that connects LLM agents to a local LM Studio instance, enabling model management, OpenAI-compatible chat completions, text completions, and embeddings through a set of tools.91-
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/setheerwagen/local-ai-mcp-servers'
If you have feedback or need assistance with the MCP directory API, please join our Discord server