mcp-ollama
Provides tools for interacting with one or more Ollama instances: listing available models, running chat completions and single-prompt generations with configurable generation options, computing embeddings, and managing long-running inference as asynchronous jobs (status, result retrieval, and cancellation). Supports multiple named backends, including local and remote HTTP(S) endpoints with HTTP Basic auth, custom CA bundles, and path prefixes.
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
mcp-ollama
An MCP (Model Context Protocol) server for one or more Ollama instances.
It exposes model discovery, chat, completion, embeddings, and asynchronous jobs as typed MCP tools. Backends are selected by name from a configuration file; local and remote HTTP(S) endpoints are supported, including endpoints behind HTTP Basic auth.
The server is designed for the case where inference is slow — a CPU-only machine or a small GPU — and an agent should not hold a tool call open for several minutes. Inference tools wait for a bounded time and return a job id when the work is still running.
Features
Several Ollama backends in one server. Local and remote instances are addressed by name; the model never supplies a URL.
HTTP Basic auth per backend. Works with Ollama behind nginx, Caddy, or another reverse proxy using an htpasswd-style user/password.
HTTPS and custom CA bundles.
verify_tlsacceptstrue,false, or a CA file.Path prefixes. A backend may live at
https://host/ollama, not only at the root of a host.Asynchronous jobs.
wait_secondsbounds a tool call; longer generations return ajob_idforollama_job_status/ollama_job_result/ollama_cancel_job.Per-backend concurrency limits. One request in flight per backend by default, which matches a small CPU-only Ollama.
Streamable HTTP and stdio. Run as a long-lived HTTP service or as a local stdio server.
Systemd-friendly. The included unit runs with
DynamicUser, a read-only filesystem apart from small state/runtime directories, no capabilities, private devices, and the backend JSON passed as a systemd credential rather than an environment variable.No telemetry. The packaged service disables FastMCP update checks and the startup banner.
Related MCP server: Ollama MCP Server
Architecture
MCP client (Claude Code, Codex, Cursor, ...)
|
| Streamable HTTP or stdio
v
mcp-ollama (FastMCP)
|
| named backend + optional Basic auth + TLS
+--> local Ollama http://127.0.0.1:11434
+--> remote Ollama https://ollama.example.com/ollama
+--> another Ollama https://other.example.comEach backend has its own httpx.AsyncClient and an asyncio.Semaphore. Inference
jobs run in the server process; the answers are held in memory for
job_ttl_seconds (one hour by default).
Requirements
Python 3.11 or newer
Ollama 0.12 or newer (tested with 0.12.11)
fastmcp4.x (tested with 4.1.7) andhttpx0.28+Linux with systemd for the included service/installer; the Python package itself runs anywhere
Quick start
git clone https://github.com/lemenkov/mcp-ollama.git
cd mcp-ollama
python3 -m venv .venv
. .venv/bin/activate
pip install -e '.[dev]'
# Make sure the default model exists locally
ollama pull llama3.2:3b
# Start the MCP server on 127.0.0.1:8815
mcp-ollama --config mcp-ollama.jsonThe Streamable HTTP endpoint is then:
http://127.0.0.1:8815/mcpFor a local stdio server instead:
mcp-ollama --config mcp-ollama.json --transport stdioConfiguration
There are two files:
File | Purpose |
| Listener and logging environment variables |
| Backend definitions and auth credentials |
The JSON file is normally root-owned and mode 0600. The systemd unit passes it to
the service with LoadCredential=, so passwords do not appear in the environment,
systemctl show, or /proc.
Listener environment
HOST=127.0.0.1
PORT=8815
OLLAMA_MCP_LOG_LEVEL=INFO
# Keep the packaged service fully local.
FASTMCP_CHECK_FOR_UPDATES=off
FASTMCP_SHOW_SERVER_BANNER=falseBackends
{
"default_backend": "local",
"backends": {
"local": {
"base_url": "http://127.0.0.1:11434",
"default_model": "llama3.2:3b",
"timeout_seconds": 600,
"max_concurrency": 1
},
"remote": {
"base_url": "https://ollama.example.com/ollama",
"username": "ollama",
"password": "change-me",
"default_model": "qwen3:4b",
"verify_tls": true,
"timeout_seconds": 600,
"max_concurrency": 1,
"description": "Remote Ollama behind nginx Basic auth"
}
},
"job_ttl_seconds": 3600,
"max_jobs": 100,
"result_max_chars": 20000
}Backend fields
Field | Default | Description |
| required |
|
|
| HTTP Basic auth user. If omitted and the URL has userinfo, the URL user is used. |
|
| HTTP Basic auth password. |
|
| Read the password from a file. Relative paths are resolved against the JSON file's directory. Use inline passwords with the systemd credential. |
|
| Model used when a tool omits |
|
|
|
|
| Read timeout for inference requests. |
|
| TCP/TLS connect timeout. |
|
| Maximum simultaneous requests to this backend. |
|
| Extra HTTP headers for this backend. |
|
| Disabled backends are not loaded and cannot be selected. |
|
| Free-form note shown by |
Top-level fields
Field | Default | Description |
| required | Backend used when a tool omits |
|
| How long completed jobs are retained in memory. |
|
| Maximum retained jobs; new submissions are rejected when the limit is reached and nothing can be evicted. |
|
| Default truncation limit for generated text returned by inference tools. |
Configuration precedence
The backend file is resolved in this order:
--config PATHOLLAMA_MCP_CONFIG$CREDENTIALS_DIRECTORY/backends.json(the systemd credential)/etc/sysconfig/mcp-ollama.json
Two different Basic auth layers
Do not confuse them:
MCP client to this server, when the server is behind a reverse proxy. The proxy checks an htpasswd file; the MCP client sends the proxy credentials.
This server to an Ollama backend, configured by
username/passwordin the backend JSON.
The MCP server has no built-in authentication. Bind it to 127.0.0.1 and put an
authenticated reverse proxy in front of it for remote clients.
Tools
Tool | Purpose |
| List configured backends and optionally probe them. |
| List models on one backend, or on all backends. |
| Chat completion with a message list. |
| Single-prompt completion. |
| Embeddings for a small number of inputs. |
| Check an asynchronous job. |
| Fetch the result of a completed job. |
| Cancel a queued or running job. |
All inference tools use the same bounded-wait model. wait_seconds defaults to 30
and is capped at 120. A value of 0 returns a job id immediately.
ollama_backends
ollama_backends(probe=True, timeout_seconds=3.0)Returns the configured backend names, their redacted URLs, and — when probe is
true — version, model count, loaded models, and latency. A backend that is down is
reported in its own error field; the call itself does not fail.
{
"default_backend": "local",
"backends": [
{
"name": "local",
"url": "http://127.0.0.1:11434/",
"reachable": true,
"version": "0.12.11",
"models_count": 1,
"loaded": [],
"latency_ms": 12.4
}
]
}Use the returned name as the backend argument of the other tools.
ollama_models
ollama_models(backend=None)With backend omitted, every enabled backend is queried. Each result contains the
model name, size, modification time, family, parameter size, and quantization:
{
"default_backend": "local",
"backends": [
{
"backend": "local",
"url": "http://127.0.0.1:11434/",
"models": [
{
"name": "llama3.2:3b",
"size": 2019393189,
"modified_at": "2026-02-08T19:04:45.19249316+01:00",
"family": "llama",
"parameter_size": "3.2B",
"quantization": "Q4_K_M"
}
]
}
]
}ollama_chat
ollama_chat(
messages,
backend=None,
model=None,
system=None,
options=None,
wait_seconds=30
)messages is a list of { "role": "system" | "user" | "assistant", "content": "..." }.
If system is supplied, do not also put a system message in messages.
options is a GenerationOptions object with any of:
Option | Range |
| 0.0 – 2.0 |
| 0.0 – 1.0 |
| 1 – 8192 |
| 512 – 32768 |
| 1 – 16 |
| 0 – 2147483647 |
| up to 8 strings |
ollama_generate
ollama_generate(
prompt,
backend=None,
model=None,
system=None,
options=None,
wait_seconds=30
)The convenient tool for rewriting, translation, and other single-prompt work.
ollama_embed
ollama_embed(input, backend=None, model=None, truncate=True, wait_seconds=30)input is a string or a list of up to 8 strings, with at most 16,000 characters in
total. Vectors are returned inline, so use a dedicated pipeline for large corpora.
truncate asks Ollama to truncate inputs longer than the model's context.
Jobs
A completed call returns a result directly:
{
"job_id": "fd5e7cda6280475b",
"kind": "generate",
"backend": "local",
"model": "llama3.2:3b",
"status": "done",
"created_at": "2026-11-10T20:13:39.616516+00:00",
"started_at": "2026-11-10T20:13:39.616560+00:00",
"finished_at": "2026-11-10T20:13:47.866026+00:00",
"elapsed_seconds": 8.25,
"result_ready": true,
"error": null,
"result": {
"backend": "local",
"response": "hello from MCP",
"model": "llama3.2:3b",
"done": true,
"done_reason": "stop",
"eval_count": 4,
"eval_duration_ns": 341321693,
"prompt_eval_count": 32,
"prompt_eval_duration_ns": 1621996874,
"total_duration_ns": 8244612415,
"load_duration_ns": 6269440872,
"tokens_per_second": 11.72
}
}If the call is still running when wait_seconds expires, the response has a job_id
and a status of queued or running:
{
"job_id": "1128ff069b2a46e6",
"kind": "generate",
"backend": "local",
"model": "llama3.2:3b",
"status": "running",
"result_ready": false,
"error": null,
"message": "Job is still running. Poll ollama_job_status, then fetch the answer with ollama_job_result."
}Typical client flow:
import asyncio
from fastmcp import Client
async def main():
async with Client("http://127.0.0.1:8815/mcp") as client:
started = await client.call_tool(
"ollama_generate",
{"prompt": "Explain systemd sandboxing in two sentences.", "wait_seconds": 0},
)
job_id = started.data["job_id"]
while True:
status = await client.call_tool("ollama_job_status", {"job_id": job_id})
if status.data["status"] in ("done", "failed", "cancelled"):
break
await asyncio.sleep(1)
result = await client.call_tool("ollama_job_result", {"job_id": job_id})
print(result.data["result"]["response"])
asyncio.run(main())ollama_job_result accepts max_chars (1,000 – 200,000, default 20,000). Long text
fields are truncated with "truncated": true and original_chars.
ollama_cancel_job cancels a queued or running job. Cancelling a finished job is
harmless.
MCP client setup
Streamable HTTP, local
{
"mcpServers": {
"ollama": {
"command": "npx",
"args": [
"mcp-remote@0.16.5",
"http://127.0.0.1:8815/mcp"
]
}
}
}Streamable HTTP behind an authenticated reverse proxy
{
"mcpServers": {
"ollama": {
"command": "npx",
"args": [
"mcp-remote@0.16.5",
"https://mcp.example.com/mcp/ollama",
"--header-file",
"/home/you/.config/mcp-auth-ollama.txt"
]
}
}
}mcp-auth-ollama.txt contains the proxy's credentials, for example:
Authorization: Basic <base64-user:password>This is the proxy credential, not the Ollama backend credential.
stdio
{
"mcpServers": {
"ollama": {
"command": "/opt/mcp-servers/mcp-ollama/venv/bin/mcp-ollama",
"args": [
"--transport", "stdio",
"--config", "/etc/sysconfig/mcp-ollama.json"
]
}
}
}A stdio server runs as the client's user and reads the JSON file directly. The
LoadCredential arrangement only applies to the systemd HTTP service.
systemd deployment
The repository includes an opinionated installer for a Fedora-like host with systemd and nginx:
sudo bash install-mcp-ollama.shReview the script before running it. It installs:
/opt/mcp-servers/mcp-ollama/ code, pyproject.toml, venv/
/etc/sysconfig/mcp-ollama listener environment
/etc/sysconfig/mcp-ollama.json backend definitions (0600)
/etc/systemd/system/mcp-ollama.service hardened unitIt also adds a /mcp/ollama proxy location to /etc/nginx/conf.d/mcp.conf, runs
nginx -t, and reloads nginx. Existing sysconfig files are preserved; replaced
units and nginx configuration are backed up first.
Manual equivalent:
sudo install -d /opt/mcp-servers/mcp-ollama
sudo cp -a mcp_ollama pyproject.toml README.md LICENSE \
/opt/mcp-servers/mcp-ollama/
sudo python3 -m venv /opt/mcp-servers/mcp-ollama/venv
sudo /opt/mcp-servers/mcp-ollama/venv/bin/pip install /opt/mcp-servers/mcp-ollama
sudo install -m 0644 mcp-ollama.sysconfig /etc/sysconfig/mcp-ollama
sudo install -m 0600 mcp-ollama.json /etc/sysconfig/mcp-ollama.json
sudo install -m 0644 mcp-ollama.service /etc/systemd/system/mcp-ollama.service
sudo systemctl daemon-reload
sudo systemctl enable --now mcp-ollama.serviceThe service is read-only apart from a small state directory
(/var/lib/mcp-ollama), a runtime directory (/run/mcp-ollama), and its private
/tmp. Its configuration is delivered via
LoadCredential=backends.json:/etc/sysconfig/mcp-ollama.json, so the JSON can stay
root-only.
Useful checks:
systemctl status mcp-ollama.service
journalctl -u mcp-ollama.service -f
# Use the PORT value from /etc/sysconfig/mcp-ollama:
ss -ltn | grep 8815nginx
If the server is already behind a server-level auth_basic, the location only needs
the proxy settings:
location ^~ /mcp/ollama {
rewrite ^/mcp/ollama(.*)$ /mcp$1 break;
proxy_set_header Accept "application/json, text/event-stream";
# FastMCP validates the Host header; keep it in its localhost allowlist.
proxy_set_header Host localhost:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Connection "";
proxy_pass http://127.0.0.1:8815;
}The proxy_pass port must match PORT in /etc/sysconfig/mcp-ollama; the
examples here use the code default, 8815.
If authentication is not inherited, add:
auth_basic "MCP Access";
auth_basic_user_file /etc/nginx/.htpasswd-mcp;Security model
Named backends only. Tools accept a backend name, never a URL, so the model cannot make the server connect to arbitrary hosts.
Credential, not environment. The backend JSON is passed as a systemd credential and is not visible in
systemctl show, the service environment, or/proc.Least privilege. The packaged unit uses
DynamicUser=yes,ProtectSystem=strict,ProtectHome=yes,PrivateDevices=yes,PrivateTmp=yes,MemoryDenyWriteExecute=yes, an empty capability set, and a@system-servicesyscall filter.Local listener. The server binds
127.0.0.1. Put an authenticated reverse proxy in front of it for remote clients.TLS for remote backends. Basic auth is only as private as the transport; use
https://for remote Ollama endpoints and keepverify_tlsenabled.No telemetry. The packaged service sets
FASTMCP_CHECK_FOR_UPDATES=offandFASTMCP_SHOW_SERVER_BANNER=false.No prompt logging by default. Logs contain backend names, model names, and timings, not message contents.
Development
python3 -m venv .venv
. .venv/bin/activate
pip install -e '.[dev]'
pytest -qThe test suite uses httpx.MockTransport, so it needs no Ollama instance and no
network. It covers:
configuration parsing, URL prefixes, userinfo, and password files
Basic auth headers and actionable HTTP errors
per-backend routing
asynchronous job submission, polling, cancellation, failure, and truncation
To run the server against a real local Ollama:
ollama pull llama3.2:3b
mcp-ollama --config mcp-ollama.jsonLimitations
Jobs are in memory. Restarting the MCP service loses running and completed jobs. The Ollama server itself keeps the loaded model, so a retry is usually cheap.
No token streaming. Inference calls use
stream=false; progress is exposed as job status, not partial tokens.Embeddings are inline.
ollama_embedis capped at 8 inputs and 16,000 characters per call. Use a dedicated batch pipeline for large corpora.No model administration. Pulling and deleting models stay in the
ollamaCLI; the MCP tools do not exposepullorrm.One request per backend by default. Raise
max_concurrencyonly if the backend has enough CPU/GPU capacity.CPU inference is slow. On a four-core Haswell, a 3B Q4 model generates roughly 10–12 tokens/s after loading. A 300-token answer takes about half a minute; a 1,000-token answer takes minutes.
License
Apache License 2.0. See LICENSE.
Copyright 2026 Peter Lemenkov.
Acknowledgements
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
Chat, image, video, and audio models from many vendors behind one remote MCP server and one key.
Use AI models for chat, image, and video generation from Claude Code and other MCP hosts.
Discover and call AI agents via MCP. Supports A2A agents and platform agents with async tasks.
Free OpenAI-compatible inference with signed provenance receipts and 3 focused MCP tools.
Related MCP Servers
- AlicenseNot gradedqualityAmaintenanceEnables MCP-compatible clients to access local or cloud LLMs, manage and run models, generate completions and embeddings, chat with tool/function support, and use web search/fetch through Ollama Cloud.812 npmAGPL 3.0
- AlicenseNot gradedqualityAmaintenanceEnables MCP-compatible clients to manage and interact with local or cloud-hosted Ollama models, including listing, pulling, generating, chatting, embeddings, and web search/fetch operations.812 npmAGPL 3.0
- AlicenseNot gradedqualityAmaintenanceEnables MCP-compatible clients such as Claude Desktop and Cline to manage and interact with local or cloud-hosted Ollama models via tools for model administration, chat, text generation, embeddings, and web search/fetch.812 npmAGPL 3.0
- AlicenseNot gradedqualityAmaintenanceEnables MCP-compatible clients to interact with local and cloud Ollama models by exposing Ollama SDK functionality as tools for model management, text generation, chat, embeddings, and web search/fetch.812 npmAGPL 3.0