Skip to main content
Glama

mcp-ollama

License: Apache-2.0 Python

An MCP (Model Context Protocol) server for one or more Ollama instances.

It exposes model discovery, chat, completion, embeddings, and asynchronous jobs as typed MCP tools. Backends are selected by name from a configuration file; local and remote HTTP(S) endpoints are supported, including endpoints behind HTTP Basic auth.

The server is designed for the case where inference is slow — a CPU-only machine or a small GPU — and an agent should not hold a tool call open for several minutes. Inference tools wait for a bounded time and return a job id when the work is still running.

Features

  • Several Ollama backends in one server. Local and remote instances are addressed by name; the model never supplies a URL.

  • HTTP Basic auth per backend. Works with Ollama behind nginx, Caddy, or another reverse proxy using an htpasswd-style user/password.

  • HTTPS and custom CA bundles. verify_tls accepts true, false, or a CA file.

  • Path prefixes. A backend may live at https://host/ollama, not only at the root of a host.

  • Asynchronous jobs. wait_seconds bounds a tool call; longer generations return a job_id for ollama_job_status / ollama_job_result / ollama_cancel_job.

  • Per-backend concurrency limits. One request in flight per backend by default, which matches a small CPU-only Ollama.

  • Streamable HTTP and stdio. Run as a long-lived HTTP service or as a local stdio server.

  • Systemd-friendly. The included unit runs with DynamicUser, a read-only filesystem apart from small state/runtime directories, no capabilities, private devices, and the backend JSON passed as a systemd credential rather than an environment variable.

  • No telemetry. The packaged service disables FastMCP update checks and the startup banner.

Related MCP server: Ollama MCP Server

Architecture

MCP client (Claude Code, Codex, Cursor, ...)
    |
    |  Streamable HTTP  or  stdio
    v
mcp-ollama  (FastMCP)
    |
    |  named backend + optional Basic auth + TLS
    +--> local Ollama       http://127.0.0.1:11434
    +--> remote Ollama      https://ollama.example.com/ollama
    +--> another Ollama     https://other.example.com

Each backend has its own httpx.AsyncClient and an asyncio.Semaphore. Inference jobs run in the server process; the answers are held in memory for job_ttl_seconds (one hour by default).

Requirements

  • Python 3.11 or newer

  • Ollama 0.12 or newer (tested with 0.12.11)

  • fastmcp 4.x (tested with 4.1.7) and httpx 0.28+

  • Linux with systemd for the included service/installer; the Python package itself runs anywhere

Quick start

git clone https://github.com/lemenkov/mcp-ollama.git
cd mcp-ollama

python3 -m venv .venv
. .venv/bin/activate
pip install -e '.[dev]'

# Make sure the default model exists locally
ollama pull llama3.2:3b

# Start the MCP server on 127.0.0.1:8815
mcp-ollama --config mcp-ollama.json

The Streamable HTTP endpoint is then:

http://127.0.0.1:8815/mcp

For a local stdio server instead:

mcp-ollama --config mcp-ollama.json --transport stdio

Configuration

There are two files:

File

Purpose

/etc/sysconfig/mcp-ollama

Listener and logging environment variables

/etc/sysconfig/mcp-ollama.json

Backend definitions and auth credentials

The JSON file is normally root-owned and mode 0600. The systemd unit passes it to the service with LoadCredential=, so passwords do not appear in the environment, systemctl show, or /proc.

Listener environment

HOST=127.0.0.1
PORT=8815
OLLAMA_MCP_LOG_LEVEL=INFO

# Keep the packaged service fully local.
FASTMCP_CHECK_FOR_UPDATES=off
FASTMCP_SHOW_SERVER_BANNER=false

Backends

{
  "default_backend": "local",
  "backends": {
    "local": {
      "base_url": "http://127.0.0.1:11434",
      "default_model": "llama3.2:3b",
      "timeout_seconds": 600,
      "max_concurrency": 1
    },
    "remote": {
      "base_url": "https://ollama.example.com/ollama",
      "username": "ollama",
      "password": "change-me",
      "default_model": "qwen3:4b",
      "verify_tls": true,
      "timeout_seconds": 600,
      "max_concurrency": 1,
      "description": "Remote Ollama behind nginx Basic auth"
    }
  },
  "job_ttl_seconds": 3600,
  "max_jobs": 100,
  "result_max_chars": 20000
}

Backend fields

Field

Default

Description

base_url

required

http:// or https:// URL. A path prefix such as /ollama is allowed. Userinfo (https://user:pass@host) is also accepted.

username

null

HTTP Basic auth user. If omitted and the URL has userinfo, the URL user is used.

password

null

HTTP Basic auth password.

password_file

null

Read the password from a file. Relative paths are resolved against the JSON file's directory. Use inline passwords with the systemd credential.

default_model

null

Model used when a tool omits model. If neither is set, the tool returns an error.

verify_tls

true

true, false, or a path to a CA bundle.

timeout_seconds

600

Read timeout for inference requests.

connect_timeout_seconds

10

TCP/TLS connect timeout.

max_concurrency

1

Maximum simultaneous requests to this backend.

headers

{}

Extra HTTP headers for this backend.

enabled

true

Disabled backends are not loaded and cannot be selected.

description

null

Free-form note shown by ollama_backends when probe=false.

Top-level fields

Field

Default

Description

default_backend

required

Backend used when a tool omits backend. Must exist and be enabled.

job_ttl_seconds

3600

How long completed jobs are retained in memory.

max_jobs

100

Maximum retained jobs; new submissions are rejected when the limit is reached and nothing can be evicted.

result_max_chars

20000

Default truncation limit for generated text returned by inference tools.

Configuration precedence

The backend file is resolved in this order:

  1. --config PATH

  2. OLLAMA_MCP_CONFIG

  3. $CREDENTIALS_DIRECTORY/backends.json (the systemd credential)

  4. /etc/sysconfig/mcp-ollama.json

Two different Basic auth layers

Do not confuse them:

  1. MCP client to this server, when the server is behind a reverse proxy. The proxy checks an htpasswd file; the MCP client sends the proxy credentials.

  2. This server to an Ollama backend, configured by username / password in the backend JSON.

The MCP server has no built-in authentication. Bind it to 127.0.0.1 and put an authenticated reverse proxy in front of it for remote clients.

Tools

Tool

Purpose

ollama_backends

List configured backends and optionally probe them.

ollama_models

List models on one backend, or on all backends.

ollama_chat

Chat completion with a message list.

ollama_generate

Single-prompt completion.

ollama_embed

Embeddings for a small number of inputs.

ollama_job_status

Check an asynchronous job.

ollama_job_result

Fetch the result of a completed job.

ollama_cancel_job

Cancel a queued or running job.

All inference tools use the same bounded-wait model. wait_seconds defaults to 30 and is capped at 120. A value of 0 returns a job id immediately.

ollama_backends

ollama_backends(probe=True, timeout_seconds=3.0)

Returns the configured backend names, their redacted URLs, and — when probe is true — version, model count, loaded models, and latency. A backend that is down is reported in its own error field; the call itself does not fail.

{
  "default_backend": "local",
  "backends": [
    {
      "name": "local",
      "url": "http://127.0.0.1:11434/",
      "reachable": true,
      "version": "0.12.11",
      "models_count": 1,
      "loaded": [],
      "latency_ms": 12.4
    }
  ]
}

Use the returned name as the backend argument of the other tools.

ollama_models

ollama_models(backend=None)

With backend omitted, every enabled backend is queried. Each result contains the model name, size, modification time, family, parameter size, and quantization:

{
  "default_backend": "local",
  "backends": [
    {
      "backend": "local",
      "url": "http://127.0.0.1:11434/",
      "models": [
        {
          "name": "llama3.2:3b",
          "size": 2019393189,
          "modified_at": "2026-02-08T19:04:45.19249316+01:00",
          "family": "llama",
          "parameter_size": "3.2B",
          "quantization": "Q4_K_M"
        }
      ]
    }
  ]
}

ollama_chat

ollama_chat(
  messages,
  backend=None,
  model=None,
  system=None,
  options=None,
  wait_seconds=30
)

messages is a list of { "role": "system" | "user" | "assistant", "content": "..." }. If system is supplied, do not also put a system message in messages.

options is a GenerationOptions object with any of:

Option

Range

temperature

0.0 – 2.0

top_p

0.0 – 1.0

num_predict

1 – 8192

num_ctx

512 – 32768

num_thread

1 – 16

seed

0 – 2147483647

stop

up to 8 strings

ollama_generate

ollama_generate(
  prompt,
  backend=None,
  model=None,
  system=None,
  options=None,
  wait_seconds=30
)

The convenient tool for rewriting, translation, and other single-prompt work.

ollama_embed

ollama_embed(input, backend=None, model=None, truncate=True, wait_seconds=30)

input is a string or a list of up to 8 strings, with at most 16,000 characters in total. Vectors are returned inline, so use a dedicated pipeline for large corpora. truncate asks Ollama to truncate inputs longer than the model's context.

Jobs

A completed call returns a result directly:

{
  "job_id": "fd5e7cda6280475b",
  "kind": "generate",
  "backend": "local",
  "model": "llama3.2:3b",
  "status": "done",
  "created_at": "2026-11-10T20:13:39.616516+00:00",
  "started_at": "2026-11-10T20:13:39.616560+00:00",
  "finished_at": "2026-11-10T20:13:47.866026+00:00",
  "elapsed_seconds": 8.25,
  "result_ready": true,
  "error": null,
  "result": {
    "backend": "local",
    "response": "hello from MCP",
    "model": "llama3.2:3b",
    "done": true,
    "done_reason": "stop",
    "eval_count": 4,
    "eval_duration_ns": 341321693,
    "prompt_eval_count": 32,
    "prompt_eval_duration_ns": 1621996874,
    "total_duration_ns": 8244612415,
    "load_duration_ns": 6269440872,
    "tokens_per_second": 11.72
  }
}

If the call is still running when wait_seconds expires, the response has a job_id and a status of queued or running:

{
  "job_id": "1128ff069b2a46e6",
  "kind": "generate",
  "backend": "local",
  "model": "llama3.2:3b",
  "status": "running",
  "result_ready": false,
  "error": null,
  "message": "Job is still running. Poll ollama_job_status, then fetch the answer with ollama_job_result."
}

Typical client flow:

import asyncio
from fastmcp import Client

async def main():
    async with Client("http://127.0.0.1:8815/mcp") as client:
        started = await client.call_tool(
            "ollama_generate",
            {"prompt": "Explain systemd sandboxing in two sentences.", "wait_seconds": 0},
        )
        job_id = started.data["job_id"]

        while True:
            status = await client.call_tool("ollama_job_status", {"job_id": job_id})
            if status.data["status"] in ("done", "failed", "cancelled"):
                break
            await asyncio.sleep(1)

        result = await client.call_tool("ollama_job_result", {"job_id": job_id})
        print(result.data["result"]["response"])

asyncio.run(main())

ollama_job_result accepts max_chars (1,000 – 200,000, default 20,000). Long text fields are truncated with "truncated": true and original_chars.

ollama_cancel_job cancels a queued or running job. Cancelling a finished job is harmless.

MCP client setup

Streamable HTTP, local

{
  "mcpServers": {
    "ollama": {
      "command": "npx",
      "args": [
        "mcp-remote@0.16.5",
        "http://127.0.0.1:8815/mcp"
      ]
    }
  }
}

Streamable HTTP behind an authenticated reverse proxy

{
  "mcpServers": {
    "ollama": {
      "command": "npx",
      "args": [
        "mcp-remote@0.16.5",
        "https://mcp.example.com/mcp/ollama",
        "--header-file",
        "/home/you/.config/mcp-auth-ollama.txt"
      ]
    }
  }
}

mcp-auth-ollama.txt contains the proxy's credentials, for example:

Authorization: Basic <base64-user:password>

This is the proxy credential, not the Ollama backend credential.

stdio

{
  "mcpServers": {
    "ollama": {
      "command": "/opt/mcp-servers/mcp-ollama/venv/bin/mcp-ollama",
      "args": [
        "--transport", "stdio",
        "--config", "/etc/sysconfig/mcp-ollama.json"
      ]
    }
  }
}

A stdio server runs as the client's user and reads the JSON file directly. The LoadCredential arrangement only applies to the systemd HTTP service.

systemd deployment

The repository includes an opinionated installer for a Fedora-like host with systemd and nginx:

sudo bash install-mcp-ollama.sh

Review the script before running it. It installs:

/opt/mcp-servers/mcp-ollama/              code, pyproject.toml, venv/
/etc/sysconfig/mcp-ollama                 listener environment
/etc/sysconfig/mcp-ollama.json            backend definitions (0600)
/etc/systemd/system/mcp-ollama.service    hardened unit

It also adds a /mcp/ollama proxy location to /etc/nginx/conf.d/mcp.conf, runs nginx -t, and reloads nginx. Existing sysconfig files are preserved; replaced units and nginx configuration are backed up first.

Manual equivalent:

sudo install -d /opt/mcp-servers/mcp-ollama
sudo cp -a mcp_ollama pyproject.toml README.md LICENSE \
    /opt/mcp-servers/mcp-ollama/
sudo python3 -m venv /opt/mcp-servers/mcp-ollama/venv
sudo /opt/mcp-servers/mcp-ollama/venv/bin/pip install /opt/mcp-servers/mcp-ollama

sudo install -m 0644 mcp-ollama.sysconfig /etc/sysconfig/mcp-ollama
sudo install -m 0600 mcp-ollama.json /etc/sysconfig/mcp-ollama.json
sudo install -m 0644 mcp-ollama.service /etc/systemd/system/mcp-ollama.service

sudo systemctl daemon-reload
sudo systemctl enable --now mcp-ollama.service

The service is read-only apart from a small state directory (/var/lib/mcp-ollama), a runtime directory (/run/mcp-ollama), and its private /tmp. Its configuration is delivered via LoadCredential=backends.json:/etc/sysconfig/mcp-ollama.json, so the JSON can stay root-only.

Useful checks:

systemctl status mcp-ollama.service
journalctl -u mcp-ollama.service -f
# Use the PORT value from /etc/sysconfig/mcp-ollama:
ss -ltn | grep 8815

nginx

If the server is already behind a server-level auth_basic, the location only needs the proxy settings:

location ^~ /mcp/ollama {
    rewrite ^/mcp/ollama(.*)$ /mcp$1 break;
    proxy_set_header Accept "application/json, text/event-stream";
    # FastMCP validates the Host header; keep it in its localhost allowlist.
    proxy_set_header Host localhost:$server_port;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Connection "";
    proxy_pass http://127.0.0.1:8815;
}

The proxy_pass port must match PORT in /etc/sysconfig/mcp-ollama; the examples here use the code default, 8815.

If authentication is not inherited, add:

auth_basic "MCP Access";
auth_basic_user_file /etc/nginx/.htpasswd-mcp;

Security model

  • Named backends only. Tools accept a backend name, never a URL, so the model cannot make the server connect to arbitrary hosts.

  • Credential, not environment. The backend JSON is passed as a systemd credential and is not visible in systemctl show, the service environment, or /proc.

  • Least privilege. The packaged unit uses DynamicUser=yes, ProtectSystem=strict, ProtectHome=yes, PrivateDevices=yes, PrivateTmp=yes, MemoryDenyWriteExecute=yes, an empty capability set, and a @system-service syscall filter.

  • Local listener. The server binds 127.0.0.1. Put an authenticated reverse proxy in front of it for remote clients.

  • TLS for remote backends. Basic auth is only as private as the transport; use https:// for remote Ollama endpoints and keep verify_tls enabled.

  • No telemetry. The packaged service sets FASTMCP_CHECK_FOR_UPDATES=off and FASTMCP_SHOW_SERVER_BANNER=false.

  • No prompt logging by default. Logs contain backend names, model names, and timings, not message contents.

Development

python3 -m venv .venv
. .venv/bin/activate
pip install -e '.[dev]'
pytest -q

The test suite uses httpx.MockTransport, so it needs no Ollama instance and no network. It covers:

  • configuration parsing, URL prefixes, userinfo, and password files

  • Basic auth headers and actionable HTTP errors

  • per-backend routing

  • asynchronous job submission, polling, cancellation, failure, and truncation

To run the server against a real local Ollama:

ollama pull llama3.2:3b
mcp-ollama --config mcp-ollama.json

Limitations

  • Jobs are in memory. Restarting the MCP service loses running and completed jobs. The Ollama server itself keeps the loaded model, so a retry is usually cheap.

  • No token streaming. Inference calls use stream=false; progress is exposed as job status, not partial tokens.

  • Embeddings are inline. ollama_embed is capped at 8 inputs and 16,000 characters per call. Use a dedicated batch pipeline for large corpora.

  • No model administration. Pulling and deleting models stay in the ollama CLI; the MCP tools do not expose pull or rm.

  • One request per backend by default. Raise max_concurrency only if the backend has enough CPU/GPU capacity.

  • CPU inference is slow. On a four-core Haswell, a 3B Q4 model generates roughly 10–12 tokens/s after loading. A 300-token answer takes about half a minute; a 1,000-token answer takes minutes.

License

Apache License 2.0. See LICENSE.

Copyright 2026 Peter Lemenkov.

Acknowledgements

  • Ollama for the local inference runtime and HTTP API.

  • FastMCP for the MCP server framework.

  • httpx for the async HTTP client.

  • Pydantic for configuration and tool schemas.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      A
      maintenance
      Enables MCP-compatible clients to access local or cloud LLMs, manage and run models, generate completions and embeddings, chat with tool/function support, and use web search/fetch through Ollama Cloud.
      812 npm
      AGPL 3.0
    • A
      license
      Not graded
      quality
      A
      maintenance
      Enables MCP-compatible clients to interact with local and cloud Ollama models by exposing Ollama SDK functionality as tools for model management, text generation, chat, embeddings, and web search/fetch.
      812 npm
      AGPL 3.0