grafana-unified-mcp

One MCP server in front of many Grafana instances. Every tool from the standard Grafana MCP server, plus one extra argument — instance — that says which Grafana to run it against.

query_prometheus(instance="tenant-a", expr="up", datasourceUid="...")
search_dashboards(instance="tenant-b", query="login latency")

Why this exists

Upstream grafana/mcp-grafana binds GRAFANA_URL once, at process start. It reads X-Grafana-Service-Account-Token per request, but the URL is fixed — and the header that used to override it is now explicitly inert. From upstream validate_url.go:

Deprecated: X-Grafana-URL no longer configures the Grafana client. This middleware is retained temporarily to preserve malformed-header handling.

So one mcp-grafana process can only ever talk to one Grafana. Ten Grafanas means ten servers, ten entries in every client config, and ten sets of tools with identical names for the model to disambiguate.

This server fixes that by running one upstream child process per instance and routing each call to the right one based on the instance argument. Tools are discovered from the real binary at runtime, so you get whatever upstream exposes — currently 65 tools — with no per-tool code here and nothing to update when upstream adds more.

How it works

                                        ┌──────────────────────────────────┐
  Claude Code / routines /              │  grafana-unified-mcp             │
  cloud sessions                        │                                  │
        │                               │  ┌────────────────────────────┐  │
        │  streamable-HTTP              │  │ bearer auth                │  │
        │  Authorization: Bearer …      │  │  → Principal(instances,    │  │
        ├──────────────────────────────►│  │      read-only|read-write) │  │
        │                               │  └────────────┬───────────────┘  │
        │                               │               │                  │
        │                               │  ┌────────────▼───────────────┐  │
        │                               │  │ catalog: inject `instance`  │  │
        │                               │  │ filter by caller's grant   │  │
        │                               │  └────────────┬───────────────┘  │
        │                               │               │ route on         │
        │                               │               │ instance=…       │
        │                               │  ┌────────────▼───────────────┐  │
        │                               │  │ child pool (lazy, reaped)  │  │
        │                               │  └──┬──────────┬──────────┬───┘  │
        └───────────────────────────────┴─────┼──────────┼──────────┼──────┘
                                              │ stdio    │ stdio    │ stdio
                                        ┌─────▼────┐ ┌───▼──────┐ ┌▼─────────┐
                                        │mcp-grafana│ │mcp-grafana│ │mcp-grafana│
                                        │ tenant-a │ │ tenant-b │ │   …      │
                                        └─────┬────┘ └───┬──────┘ └┬─────────┘
                                              ▼          ▼         ▼
                                          tenant-a    tenant-b   …Grafana

Children start on first use, stay warm, get reaped when idle (--idle-timeout, default 15 min), and are respawned transparently if they die. An unreachable Grafana degrades only its own instance.

How to install

Docker is the supported path. The image builds the upstream mcp-grafana binary itself, so the host needs no Go toolchain and no Python — just Docker.

1. Ports — what you actually need to open

One port, however many Grafana instances you configure. This is the part people over-plan: the per-instance mcp-grafana children are stdio subprocesses talking over pipes, so they consume no ports and need nothing published. Ten Grafanas and one Grafana look identical from the network.

Nothing ever connects inward from Grafana. There is no second listener, no per-instance port, and no port-range requirement.

The container listens on 21000 internally. Map whichever host port you want from your 21000–21999 range:

HOST_PORT=21000    # ELB target group points here

2. Build and run

git clone https://github.com/robert-sinclair/grafana-unified-mcp.git
cd grafana-unified-mcp

# Pin the upstream ref — it decides which tool catalogue gets republished.
export MCP_GRAFANA_REF=main
export HOST_PORT=21000
export CONFIG_DIR=/opt/grafana-unified-mcp/config
export PUBLIC_URL=https://grafana-mcp.example.com   # your ELB's URL

docker compose up -d --build
curl -s localhost:${HOST_PORT}/healthz | jq

The build takes a few minutes the first time — it compiles mcp-grafana from source and then verifies the binary publishes its tools before the image is accepted, so a broken upstream build fails here rather than at deploy.

Plain Docker, without compose:

docker build --build-arg MCP_GRAFANA_REF=main -t grafana-unified-mcp:local .

docker run -d --name grafana-unified-mcp --init --restart unless-stopped \
  -p 21000:21000 \
  -v /opt/grafana-unified-mcp/config:/etc/grafana-unified-mcp:ro \
  -e MCP_PUBLIC_URL=https://grafana-mcp.example.com \
  grafana-unified-mcp:local

--init matters: this process supervises one child per Grafana instance, so PID 1 has to reap them. Compose sets init: true for you.

3. Config, and the one permission that bites

Render endpoints.json and auth.json into CONFIG_DIR on the host and mount them read-only. Do not copy them into the image — the endpoints document holds a service-account token per Grafana, and an image layer is readable by anyone who can pull it. The .dockerignore excludes them so this cannot happen by accident.

The container runs as UID 10001, and a bind mount keeps the host's ownership. Root-owned 0640 files — exactly what an Ansible template task produces by default — are unreadable inside the container and the server exits at startup. So have Ansible finish with:

- name: Render unified MCP config
  ansible.builtin.template:
    src: "{{ item }}.j2"
    dest: "/opt/grafana-unified-mcp/config/{{ item }}"
    owner: "10001"          # matches the container user
    group: "10001"
    mode: "0640"            # readable in-container, not world-readable on host
  loop: [endpoints.json, auth.json]
  no_log: true              # the rendered files contain Grafana tokens
  notify: restart grafana-unified-mcp

UID 10001 maps to no real account on the host, so the files stay root-only there.

You do not need to restart to add or remove an instance — both documents are re-read on an interval (--config-refresh-seconds, default 300). A restart is only needed for changes to flags or the image.

4. Wire up the ELB

Listener 443 (HTTPS) → target group HTTP on your host port. TLS terminates at the load balancer, so the container serves plain HTTP and there is no cert to mount.

Four settings are not the defaults, and each one fails in a way that looks like something else:

Health-check specifics: protocol HTTP, path /healthz, port "traffic port", success codes 200.

If you run more than one target, set MCP_STATELESS=true. MCP session state is in-memory, so with two containers a client's follow-up request can land on the one that has never heard of its session. ALB cookie stickiness is not a reliable fix, because MCP clients are not browsers and need not send cookies. One target, or stateless — those are the two correct configurations.

Also consider a short target-group deregistration delay: on redeploy, in-flight SSE streams are cut, and clients reconnect.

Running it without Docker

The container is the recommended path, but the server is a normal Python package. It needs Python 3.11+ and the upstream binary on PATH:

deploy/install-mcp-grafana.sh /usr/local/bin   # needs Go 1.21+, GOTOOLCHAIN=auto does the rest
python3 -m venv /opt/grafana-unified-mcp/.venv
/opt/grafana-unified-mcp/.venv/bin/pip install 'grafana-unified-mcp[aws] @ .'

deploy/ also carries a hardened systemd unit and an nginx example if you would rather not use containers at all. Point at an existing binary with MCP_GRAFANA_BINARY=/path/to/mcp-grafana.

Configure

Endpoints

Exactly the shape you'd expect — instance name to the upstream env vars:

{
  "tenant-a": {
    "GRAFANA_URL": "https://tenant-a.example.cloud/grafana",
    "GRAFANA_SERVICE_ACCOUNT_TOKEN": "glsa_…"
  },
  "tenant-b": {
    "GRAFANA_URL": "https://tenant-b.example.cloud/grafana",
    "GRAFANA_SERVICE_ACCOUNT_TOKEN": "glsa_…",
    "description": "Tenant B production"
  }
}

Optional per-instance keys: GRAFANA_ORG_ID, GRAFANA_USERNAME / GRAFANA_PASSWORD, description, extra_env, extra_args. To keep secrets out of the document itself, use GRAFANA_SERVICE_ACCOUNT_TOKEN_ENV (read from this process's environment) or GRAFANA_SERVICE_ACCOUNT_TOKEN_FILE (a path the child reads).

Auth

{
  "clients": [
    {
      "name": "claude-routines",
      "token_sha256": "3f786850e387550fdab836ed7e6dc881de23001b…",
      "instances": ["tenant-a", "tenant-b"],
      "scope": "read-only"
    },
    {
      "name": "platform-oncall",
      "token_sha256": "…",
      "instances": ["*"],
      "scope": "read-write"
    }
  ]
}

Mint a token and its hash:

grafana-unified-mcp --hash-token          # generates one
grafana-unified-mcp --hash-token 'my-existing-token'

Give token to the client; put token_sha256 in the document. Tokens are compared by digest under hmac.compare_digest, and every client is checked on every attempt so match position doesn't leak through timing.

Two things are enforced per caller:

• instances — the instance enum a caller sees is narrowed to its grant, and a call naming an instance outside it is refused with the same message as a nonexistent one, so a token can't enumerate what it can't reach.

• scope — read-only callers never even see mutating tools. The split comes from upstream's own readOnlyHint annotation (49 of 65 tools are read-only today), not a list maintained here, so tools added upstream are classified without a code change. Anything unannotated is treated as not read-only.

For belt-and-braces, add --child-arg=--disable-write to strip write tools at the source for every caller.

Running without authentication

--auth-mode none serves every caller that can reach the port, read-only. There is no identity to scope instances by, so all configured instances stay readable — but nothing is writable, because an open port should not be able to rewrite a dashboard or delete a snapshot. That's enforced at three layers:

1. the published catalogue omits every mutating tool;

2. the authorization check refuses them even if a client names one directly;

3. children are started with --disable-write, so upstream refuses them too.

The third layer is what makes it more than a filter. Upstream swaps grafana_api_request for a separate GET-only registration — no body parameter, method narrowed to GET, non-GET rejected at runtime — so even a bug in layers 1 and 2 could not turn into a write.

stdio is different: the local caller already holds the endpoints document and every token in it, so restricting them would be theatre. stdio gets full access.

If you need writes over HTTP, use bearer tokens with a read-write client rather than an open port.

Where config comes from

Any of these, for both --endpoints and --auth:

Both documents are re-read every --config-refresh-seconds (default 300). A failed refresh logs and keeps the last good value, so a transient AWS error or a half-written file can't take the server down. Adding an instance needs no restart; removing one stops its child.

Validate before starting:

grafana-unified-mcp --endpoints … --auth … --check-config

Run

In the container everything is already wired (see How to install). Running it directly:

# local, over stdio (no auth — the local caller already holds the config)
grafana-unified-mcp --endpoints ./examples/endpoints.json

# streamable-HTTP, reading config from AWS
grafana-unified-mcp \
  --transport streamable-http \
  --address 0.0.0.0:21000 \
  --endpoints aws-secrets:prod/grafana/endpoints?region=us-west-2 \
  --auth      aws-secrets:prod/grafana/mcp-auth?region=us-west-2 \
  --public-url https://grafana-mcp.example.com

Every transport flag has an environment fallback, so the image needs no entrypoint script: MCP_TRANSPORT, MCP_ADDRESS, MCP_PUBLIC_URL, MCP_ALLOWED_HOSTS (comma-separated), MCP_AUTH_MODE, MCP_STATELESS, plus GRAFANA_ENDPOINTS_SOURCE and MCP_AUTH_SOURCE. Run --help for the rest.

GET /healthz reports process health, live children, and catalog state without touching Grafana — which is why it is the right target-group health check.

Connect a client

.mcp.json, for local stdio use:

{
  "mcpServers": {
    "grafana": {
      "command": "/opt/grafana-unified-mcp/.venv/bin/grafana-unified-mcp",
      "args": ["--endpoints", "/etc/grafana-unified-mcp/endpoints.json"]
    }
  }
}

For the deployed server — including Claude Code routines and cloud sessions, which is the case the bearer tokens exist for:

{
  "mcpServers": {
    "grafana": {
      "type": "http",
      "url": "https://grafana-mcp.example.com/mcp",
      "headers": {
        "Authorization": "Bearer ${GRAFANA_UNIFIED_MCP_TOKEN}"
      }
    }
  }
}

Set GRAFANA_UNIFIED_MCP_TOKEN in the environment the session runs in — for Claude Code on the web, that's the environment's variables, so scheduled routines and cloud sessions pick it up without the secret living in the repo. Give routines a read-only client; keep read-write for humans.

Deploying without containers

deploy/ carries a hardened systemd unit (ProtectSystem=strict, PrivateTmp, NoNewPrivileges, empty CapabilityBoundingSet), an installer, and an nginx example:

sudo deploy/install.sh                       # user, dirs, venv, unit file
sudo systemctl edit grafana-unified-mcp      # set the source URIs / region
sudo systemctl enable --now grafana-unified-mcp

The nginx example disables response buffering, which is required for SSE streaming — the same reason the ELB needs a raised idle timeout.

Using it

Point the model at list_grafana_instances first:

list_grafana_instances()
→ { "instances": [ {"name": "tenant-a", "url": "…", "connection": "live"}, … ],
    "routing_argument": "instance",
    "access": { "client": "claude-routines", "scope": "read-only" } }

Then every other tool takes that name:

search_dashboards(instance="tenant-a", query="latency")

Pass check_health=true to also probe each Grafana — slower, since it opens a connection to every instance.

One naming wrinkle

Upstream's grafana_api_request already has a required parameter called endpoint (the API path). Injecting a routing argument by that name would silently shadow it, which is why the routing argument is instance by default. If you rename it with --routing-param endpoint, that tool's own parameter is automatically republished as api_path and mapped back on the way through — no tool is ever broken by the collision, whatever you choose.

Development

uv venv && uv pip install -e '.[dev,aws]'
uv run pytest                       # unit + integration

The integration tests drive a real mcp-grafana child against an unreachable Grafana: enough to prove catalog discovery, instance injection and stripping, routing, and auth filtering, without needing live credentials. Set MCP_GRAFANA_BINARY to point at the binary, or they skip.

Roadmap

• OAuth 2.1 — the auth layer is already an interface, and the SDK already takes an OAuth provider alongside the token verifier. Filling in OAuth2Provider.verify_token is the whole job; auth/oauth.py documents the three steps. Map IdP groups onto the existing grafana:read / grafana:write / instance:<name> scopes and every authorization check keeps working unchanged.

• Fan-out — instance: "*" to run one read-only query across every instance and merge results. Useful for "which of these is alerting?"; left out for now because result merging deserves its own design.