# prom-evidence-mcp

An evidence-first Prometheus [MCP](https://modelcontextprotocol.io) server. It lets an AI agent query production telemetry, but every response carries the receipts: the exact PromQL that ran, when it ran, how many series and samples came back, and explicit warnings when the data is empty, high cardinality, or too thin to support a trend.

## Why

I spent years doing production incident response on systems processing over a million orders a day, and the lesson that stuck is that the most dangerous thing in an incident is not missing data, it is a confident conclusion built on partial data. Agents make this worse: they are fluent enough to turn three data points into a convincing root cause story.

This server takes a position: **an agent should never be able to read telemetry without also receiving the evidence needed to doubt it.** Concretely:

- An empty result comes back with a warning that absence of data is not evidence of absence, and a pointer to verify the metric name first.
- A range query with too few samples per series says plainly that trends computed on it are unreliable.
- A high cardinality result warns that aggregates over hundreds of series are easy to misread.
- `target_health` exists so the agent can check whether the scrapes behind the numbers are even alive before reasoning about the numbers.

The evidence block is part of the tool contract, so a well-prompted agent cites its queries and coverage the same way a careful engineer pastes the graph link into the incident channel.

## Tools

| Tool | What it does |
| --- | --- |
| `instant_query` | Evaluate a PromQL expression now (or at a timestamp), with evidence |
| `range_query` | Evaluate over a window with a step, with sample-coverage warnings |
| `list_metrics` | List or filter metric names, for verifying a metric exists before trusting an empty result |
| `target_health` | Up/down summary per scrape job, with last errors for down targets |

Every response is JSON with `result` plus an `evidence` object:

```json
{
  "evidence": {
    "query": "sum(rate(http_requests_total[5m]))",
    "endpoint": "/api/v1/query_range",
    "executed_at": "2026-07-11T22:41:03.512Z",
    "window": { "start": "2026-07-11T20:00:00Z", "end": "2026-07-11T22:00:00Z", "step": "60s" },
    "duration_ms": 84,
    "series_returned": 1,
    "total_samples": 121,
    "warnings": []
  }
}
```

## Install

```bash
git clone https://github.com/elizabethpammi/prom-evidence-mcp
cd prom-evidence-mcp
npm install
npm run build
```

## Use with Claude Code

```bash
claude mcp add prometheus --env PROMETHEUS_URL=https://your-prometheus.example.com -- node /path/to/prom-evidence-mcp/dist/index.js
```

## Use with Claude Desktop

```json
{
  "mcpServers": {
    "prometheus": {
      "command": "node",
      "args": ["/path/to/prom-evidence-mcp/dist/index.js"],
      "env": { "PROMETHEUS_URL": "https://your-prometheus.example.com" }
    }
  }
}
```

No Prometheus handy? Point it at the public PromLabs demo instance to try it out:

```bash
PROMETHEUS_URL=https://demo.promlabs.com npm run smoke
```

## Configuration

| Env var | Default | Meaning |
| --- | --- | --- |
| `PROMETHEUS_URL` | required | Base URL of the Prometheus server |
| `PROM_TIMEOUT_MS` | `15000` | Per-request timeout |
| `PROM_MAX_SERIES` | `200` | Series count above which results carry a cardinality warning |
| `PROM_MIN_RANGE_SAMPLES` | `5` | Samples per series below which range results carry a reliability warning |

## Non-goals

This is a read-only window onto telemetry. It deliberately does not expose admin endpoints, remote write, or configuration reload. An agent that can read your metrics should not be one typo away from changing them.

## License

MIT