pinpoint-apm-mcp
by Karma098
README.md
# Pinpoint APM MCP Server
[](./LICENSE)



**pinpoint-apm-mcp is an MCP server for [Pinpoint APM](https://github.com/pinpoint-apm/pinpoint): a read-only, guardrailed [Model Context Protocol](https://modelcontextprotocol.io) bridge that lets Claude and other AI agents debug production using Pinpoint traces, metrics, transactions, and service topology.**
pinpoint-mcp lets AI assistants such as Claude, or any MCP client that speaks Streamable HTTP, investigate production incidents using your existing Pinpoint Web deployment. They can check application health, search transactions, read redacted traces, compare agent metrics, inspect service dependencies and, when you allow it, take live thread dumps. Strict limits make sure an AI can never overload, change, or leak data from Pinpoint.
> This project is independent and is not affiliated with or endorsed by the Pinpoint project or NAVER.
---
## Table of contents
- [What it provides](#what-it-provides)
- [How it works](#how-it-works)
- [Requirements](#requirements)
- [Getting started](#getting-started)
- [Connecting MCP clients](#connecting-mcp-clients)
- [Tools reference](#tools-reference)
- [Response format](#response-format)
- [Error codes](#error-codes)
- [Guardrails](#guardrails)
- [Configuration](#configuration)
- [Authentication](#authentication)
- [Pinpoint routes used](#pinpoint-routes-used)
- [Health, readiness, and logging](#health-readiness-and-logging)
- [Caching and scaling](#caching-and-scaling)
- [Model system prompt and evaluations](#model-system-prompt-and-evaluations)
- [Deployment](#deployment)
- [Production rollout checklist](#production-rollout-checklist)
- [Troubleshooting](#troubleshooting)
- [Limitations](#limitations)
- [Development](#development)
- [Contributing](#contributing)
- [Security](#security)
- [License](#license)
---
## What it provides
| Capability | What the AI can do |
|---|---|
| **Discovery** | Find applications by partial name, browse the inventory page by page, and see which optional modules and limits are active. |
| **Application health** | Read response-time histogram counts, error and slow counts, and Apdex for a short window. "No traffic" is kept separate from "missing data". |
| **Agents** | List an application's agents, look at one agent, and page through lifecycle events (connect, shutdown, unexpected close, deadlock, and so on), with agent pings optionally hidden. |
| **Metrics** | Application, agent, URI, and system (host) metrics as exact time series, derived summaries, a two-agent comparison, or a URI ranking. |
| **Transactions** | Raw scatter dots, filtered transaction rows (failed, slow, by agent, by endpoint), or grouped summaries. |
| **Traces** | A typed summary of one trace (slowest calls, exceptions, SQL fingerprints, external/RPC calls) or pages of redacted call-stack events. |
| **Topology** | Ranked one-hop callers and callees with exact call, error, slow, average, and maximum values. |
| **Live diagnostics** *(optional)* | Active-thread counts across live agents, plus light or full thread dumps for one agent. |
### Design principles
- **Read-only.** The server can only reach a fixed list of verified Pinpoint Web `GET` routes. It has no admin, alarm, config, or write access, and no general HTTP passthrough.
- **Evidence first.** Exact modes return Pinpoint values unchanged, apart from mandatory redaction. Derived modes are clearly labelled. Data is never silently dropped: a partial result always comes with a cursor or a recovery action.
- **Honest outcomes.** *No traffic*, *no metric samples*, *capability unavailable*, *upstream failure*, and *partial evidence* are separate results. None of them is reported as a healthy zero.
- **Ask, don't guess.** When a call is missing context, or asks for too much, the server returns `needs_input` with one question and a safe smaller query before it touches Pinpoint.
- **Telemetry is data, not instructions.** Trace annotations and messages can never add applications or tool arguments.
---
## How it works
```mermaid
flowchart LR
C[MCP client / AI assistant] -- "HTTPS + Bearer JWT" --> G[Ingress / TLS]
G --> H["Express app — /mcp"]
H --> A["Host/Origin checks, 64 KB body limit, Ed25519 JWT verify"]
A --> T["Tool layer — strict schemas, rate limits, 20 s deadline, output ceiling"]
T --> S["Services — time windows, signed cursors, ownership checks, aggregation, coverage"]
S --> P["Pinpoint client — route allowlist, semaphore, timeouts, retry, body limits, redaction, caches"]
P -- "GET, JSON only" --> W[(Pinpoint Web 2.5.4)]
```
1. **Transport.** Streamable HTTP at `/mcp`. Each HTTP request gets a fresh `McpServer`. The Pinpoint client, caches, concurrency semaphore, and rate limiter are shared by the whole process.
2. **Authentication.** Every `/mcp` request needs a bearer JWT signed offline with Ed25519. The server has only the public key.
3. **Tool layer.** Input is checked against strict Zod schemas that reject unknown fields. The layer applies per-caller rate limits, a 20-second deadline per call, and a hard limit on output size.
4. **Services.** Resolve and bound time windows, check that agents and traces belong to the requested application, paginate with HMAC-signed cursors, build summaries, and report coverage.
5. **Pinpoint client.** Can only build URLs for allowlisted routes. It limits concurrent upstream requests, applies timeouts and body-size limits, refuses redirects, retries transient failures once, and redacts sensitive fields before anything is cached or returned.
Source layout:
```
src/
main.ts # process entry: config, client, readiness loop, HTTP listener
http.ts # Express app, health endpoints, bearer middleware, safe error handler
auth.ts # Ed25519 JWT verifier
config.ts # environment parsing and validation (fails closed)
schemas.ts # tool input and output schemas
guardrails.ts # semaphore, deadlines, bounded body readers, TTL cache
runtime.ts # response envelope, error codes, signed cursors, rate limiter, output ceiling
tools.ts # builds the MCP server for one request
tooling/ # tool registration (workflow, live, and an internal granular profile)
services/ # discovery, health, agents, metrics, transactions, traces, topology, live
pinpoint/ # route allowlist, HTTP/WebSocket client, normalizers, redaction
scripts/mint-token.mjs # offline JWT minting
deploy/ # Dockerfile and Helm chart
tests/ # node:test unit and MCP/HTTP integration tests
evals.xml # agent evaluation scenarios
SYSTEM_PROMPT.md # recommended model instructions
```
---
## Requirements
| Component | Version / note |
|---|---|
| Node.js | 22.19 or newer |
| Pinpoint Web | 2.5.4. The adapter is built against this version's route shapes; other versions may return different shapes (`UPSTREAM_SCHEMA_CHANGED`). |
| Network | The server must reach Pinpoint Web and get JSON back **without an interactive login**. |
| Safe application | One existing Pinpoint application that startup preflight probes can query. |
| TLS | `MCP_PUBLIC_URL` must be HTTPS. Terminate TLS at an ingress or reverse proxy. |
| Docker | Optional, for image builds. |
| Kubernetes and Helm | Optional: Kubernetes 1.27+ and Helm 3. |
---
## Getting started
### 1. Install and verify
```bash
git clone https://github.com/Karma098/pinpoint-apm-mcp.git
cd pinpoint-apm-mcp
npm ci
npm run check # strict TypeScript build + full test suite
```
### 2. Create a signing key pair
```bash
openssl genpkey -algorithm ED25519 -out jwt-private.pem
openssl pkey -in jwt-private.pem -pubout -out jwt-public.pem
chmod 600 jwt-private.pem
```
Keep `jwt-private.pem` offline. Only the public key goes to the server. `*.pem` files are already gitignored.
### 3. Configure
```bash
cp configuration.example .env
```
Edit `.env` and set at least these values:
```bash
PINPOINT_BASE_URL=http://pinpoint-web.internal:8080
PINPOINT_SAFE_APPLICATION=my-safe-app
MCP_PUBLIC_URL=https://pinpoint-mcp.example.com/mcp
MCP_ALLOWED_HOSTS=pinpoint-mcp.example.com
MCP_JWT_PUBLIC_KEY_PEM="-----BEGIN PUBLIC KEY-----
...contents of jwt-public.pem...
-----END PUBLIC KEY-----"
MCP_CURSOR_HMAC_SECRET=<output of: openssl rand -base64 48>
```
See [Configuration](#configuration) for every option.
### 4. Run
```bash
set -a; . ./.env; set +a
npm run build
npm start # or: npm run dev (watch mode)
```
The server listens on `MCP_HTTP_HOST:MCP_HTTP_PORT` (default `0.0.0.0:3000`) and runs a startup preflight against Pinpoint.
```bash
curl -s http://localhost:3000/health/live # {"status":"ok"}
curl -s http://localhost:3000/health/ready # {"status":"ready"} once preflight passes
```
Put a TLS proxy or ingress in front of it that serves `MCP_PUBLIC_URL`.
### 5. Mint a token
```bash
MCP_PUBLIC_URL=https://pinpoint-mcp.example.com/mcp \
JWT_SUBJECT="alice" \
JWT_PRIVATE_KEY_FILE=./jwt-private.pem \
npm run mint-token
```
| Variable | Required | Notes |
|---|---|---|
| `MCP_PUBLIC_URL` | yes | Becomes the token `aud`. Must exactly match the server's value. |
| `JWT_SUBJECT` | yes | Caller identity (`sub`). Rate limits and cursors are tied to it. |
| `JWT_PRIVATE_KEY_FILE` | yes | Path to the Ed25519 PKCS#8 private key. |
| `JWT_ISSUER` | no | Default `pinpoint-mcp-local`. Must match the server's `JWT_ISSUER`. |
| `JWT_TTL_SECONDS` | no | 60 – 31536000. Default one year. |
---
## Connecting MCP clients
Every client must send `Authorization: Bearer <token>` to `MCP_PUBLIC_URL`.
### Claude Code
```bash
claude mcp add --transport http pinpoint https://pinpoint-mcp.example.com/mcp \
--header "Authorization: Bearer $PINPOINT_MCP_TOKEN"
```
### Claude Desktop and other stdio-only clients (via `mcp-remote`)
```json
{
"mcpServers": {
"pinpoint": {
"command": "npx",
"args": [
"mcp-remote",
"https://pinpoint-mcp.example.com/mcp",
"--header",
"Authorization:${PINPOINT_AUTH}"
],
"env": { "PINPOINT_AUTH": "Bearer <token>" }
}
}
}
```
### MCP Inspector (CLI)
```bash
read -rs MCP_TOKEN && export MCP_TOKEN # keeps the token out of shell history
export MCP_PUBLIC_URL=https://pinpoint-mcp.example.com/mcp
# list tools
npx @modelcontextprotocol/inspector --cli "$MCP_PUBLIC_URL" \
--transport http --method tools/list \
--header "Authorization: Bearer $MCP_TOKEN"
# call a tool
npx @modelcontextprotocol/inspector --cli "$MCP_PUBLIC_URL" \
--transport http --method tools/call \
--header "Authorization: Bearer $MCP_TOKEN" \
--tool-name pinpoint_discover --tool-arg view=capabilities
```
Never expose the Inspector proxy to an untrusted network.
> **Tip:** use [SYSTEM_PROMPT.md](./SYSTEM_PROMPT.md) as the model's system instructions. See [below](#model-system-prompt-and-evaluations).
---
## Tools reference
Every tool is annotated `readOnlyHint: true`, `destructiveHint: false`, and `openWorldHint: false`. All tools return the same [response envelope](#response-format).
### Common input types
**`application_ref`** is an exact Pinpoint application name, 1–200 characters. URLs, `?`, `&`, `#`, and control characters are rejected.
**`time`** takes one of two forms:
```jsonc
{ "kind": "relative", "minutes": 5 } // 1–1440; ends now
{ "kind": "absolute",
"start": "2026-09-08T10:10:00+05:30", // ISO-8601 with offset
"end": "2026-09-08T10:20:00+05:30" }
```
A window must start before it ends and cannot end in the future (up to 30 seconds of clock skew is tolerated). If a window is longer than the tool allows, you get `needs_input` with a suggested window of the maximum length. For relative windows the suggestion ends now; for absolute windows it keeps your requested end time.
**`cursor`** is the opaque `next_cursor` from an earlier response. Where noted, you can pass it **on its own** to continue the same fixed query.
### Tool overview
| Tool | Purpose | Max window | Rate class |
|---|---|---|---|
| [`pinpoint_discover`](#pinpoint_discover) | Capabilities and application lookup | — | normal |
| [`pinpoint_health`](#pinpoint_health) | Histogram and Apdex health | 15 min | expensive |
| [`pinpoint_agents`](#pinpoint_agents) | Agent list, detail, events | 24 h (events) | normal / expensive (events) |
| [`pinpoint_metrics`](#pinpoint_metrics) | Metric catalog and queries | 24 h | normal (catalog) / expensive (query) |
| [`pinpoint_transactions_search`](#pinpoint_transactions_search) | Scatter, items, summaries | 30 min | expensive |
| [`pinpoint_trace`](#pinpoint_trace) | Trace summary and events | fixed by trace | expensive |
| [`pinpoint_topology`](#pinpoint_topology) | One-hop dependencies | 30 min | expensive |
| [`pinpoint_active_threads`](#pinpoint_active_threads) | Live active-thread counts | — | live |
| [`pinpoint_thread_dump`](#pinpoint_thread_dump) | Live thread dump | — | dump |
---
### `pinpoint_discover`
Returns the server's capabilities, or finds applications. It never pulls telemetry.
| Parameter | Type | Default | Notes |
|---|---|---|---|
| `view` | `capabilities` \| `applications` | — | Required. |
| `query` | string (1–100) | — | Case-insensitive partial name. Omit it, or pass `*`, to browse. |
| `page_size` | int 1–5 | 5 | |
| `cursor` | string | — | Continues browsing. |
- `capabilities` returns the Pinpoint version, the tool list, module flags (`uri_metrics`, `system_metrics`, `live_active_threads`, `light_thread_dump`, `full_thread_dump`), and the enforced limits.
- `applications` with exactly one match returns that application plus a suggested `pinpoint_health` follow-up. Several matches return `needs_input` with up to five candidates. No match returns `needs_input` explaining that the search found nothing.
```json
{ "view": "applications", "query": "checkout" }
```
### `pinpoint_health`
Start here for a broad question about one application.
| Parameter | Type | Default | Notes |
|---|---|---|---|
| `application_ref` | string | — | If missing, the server asks for it. |
| `current` | boolean | — | `true` means the last 5 minutes. |
| `time` | time | — | Up to 15 minutes. Use either `current` or `time`, not both. |
| `focus` | `overview` \| `latency` \| `errors` | `overview` | Decides the suggested follow-up. |
Returns the response-time histogram from `/getResponseTimeHistogramDataV2` (totals, error and slow counts, buckets, `Avg`/`Max`/`Sum`/`Tot`) and Apdex from `/getApdexScore`. When there are no observations, or one of the two sources has no samples, the result is `partial` with the reason `no_observations`, `missing_apdex_samples`, or `missing_histogram_samples`. Results are cached for 15 seconds, and a cached result says so in its warnings. The suggested follow-up is one scatter, failed-transaction, or slow-transaction search in the same fixed window.
```json
{ "application_ref": "prod:checkout", "current": true, "focus": "errors" }
```
### `pinpoint_agents`
| Parameter | Type | Default | Notes |
|---|---|---|---|
| `view` | `list` \| `detail` \| `events` | — | Required. |
| `application_ref` | string | — | |
| `agent_id` | string | — | Required for `detail` and `events`. |
| `time` | time | — | Required for `events`, up to 24 hours. |
| `exclude_pings` | boolean | `false` | Leaves out Pinpoint ping events (code `10199`). |
| `page_size` | int 1–50 | 20 | |
| `cursor` | string | — | |
The server confirms that an agent belongs to the application. If the agent is temporarily missing from the application's agent list, it checks `/getAgentInfo` and requires the exact `applicationName` to match. Event pages keep source timestamps and event codes.
```json
{ "view": "events", "application_ref": "prod:checkout", "agent_id": "checkout-01",
"time": { "kind": "relative", "minutes": 30 }, "exclude_pings": true }
```
### `pinpoint_metrics`
| Parameter | Type | Default | Notes |
|---|---|---|---|
| `view` | `catalog` \| `query` | — | Start with `catalog` to get valid metric IDs. |
| `target` | object | — | See the target kinds below. |
| `time` | time | — | Required for `query`, up to 24 hours. |
| `metric_ids` | string[] 1–4 | — | Required for `query`. |
| `mode` | `series` \| `summary` \| `comparison` \| `ranking` | `series` | |
| `max_points` | int 1–120 | 60 | |
| `top_k` | int 1–10 | 5 | Used by ranking. |
| `cursor` | string | — | Only for `series`. Can be passed on its own. |
**Target kinds:**
| `kind` | Fields | Notes |
|---|---|---|
| `application` | `application_ref` | Metrics: `cpuLoad`, `memory`, `transaction`, `activeTrace`, `responseTime`, `dataSource`, `fileDescriptor`, `directBuffer`, `totalThreadCount`, `loadedClass`. |
| `agent` | `application_ref`, `agent_ids` (1–2) | Metrics: `jvmGc`, `jvmGcDetailed`, `cpuLoad`, `transaction`, `activeTrace`, `dataSource`, `responseTime`, `deadlock`, `fileDescriptor`, `directBuffer`, `totalThreadCount`, `loadedClass`, plus the alias `memory`, which is the `JVM_MEMORY_*` series from `jvmGc`. At most 4 agent × metric pairs per call. |
| `uri` | `application_ref`, optional `agent_id`, optional `uri` | Needs the URI module. Without `uri`: `ranking` over one `apdex`, `count`, or `time` metric. With `uri`: `uri.chart.total` or `uri.chart.failure` as a series or summary. |
| `system` | `application_ref`, `host_group`, `host`, optional `metric_id`, optional `tags` (≤ 20) | Needs the system-metric module. |
**Modes:**
- **`series`** (exact): chronological source values, including Pinpoint's `-1` "not collected" marker. Points are never thinned out. If a page is partial, continue with `next_cursor`, which keeps the same fixed window. Don't draw whole-window conclusions from a partial page.
- **`summary`** (derived): per source column it gives the minimum, maximum, and latest value with timestamps (`minimum_at`, `maximum_at`). `-1` values are left out and counted as `missing_samples`. Application summaries keep Pinpoint's own min/max agent pairing in `source_extrema_agents`.
- **`comparison`** (derived): exactly **two agents and one metric**. The result is `partial` when samples are missing or the two sides have different sample counts (`unequal_sample_coverage`).
- **`ranking`**: only for URI metrics that Pinpoint itself sorts.
An application chart that is valid but empty returns `no_metric_samples`, never zero.
```json
{ "view": "query",
"target": { "kind": "agent", "application_ref": "prod:checkout", "agent_ids": ["checkout-01", "checkout-02"] },
"time": { "kind": "relative", "minutes": 15 },
"metric_ids": ["cpuLoad"], "mode": "comparison" }
```
### `pinpoint_transactions_search`
| Parameter | Type | Default | Notes |
|---|---|---|---|
| `application_ref` | string | — | |
| `time` | time | — | Up to 30 minutes. |
| `status` | `all` \| `success` \| `failed` | `all` | |
| `latency_ms_min` | int 0–3600000 | 0 | |
| `latency_ms_max` | int 1–3600000 | — | Must be ≥ `latency_ms_min`. |
| `agent_id` | string | — | |
| `endpoint_prefix` | string | — | |
| `output` | `items` \| `scatter` \| `summary` | `items` | |
| `group_by` | `endpoint` \| `error_class` \| `latency_bucket` \| `agent` | — | For `summary`. |
| `page_size` | int 1–50 | 8 items / 20 others | Exact item pages are capped at **8**. Asking for more returns `needs_input` with a working smaller page. |
| `cursor` | string | — | Pass it on its own to continue. Not supported for `summary`. |
- **`items`** returns exact filtered transaction rows, each with a signed `trace_ref` for `pinpoint_trace`.
- **`scatter`** returns Pinpoint's raw dot arrays. The field order is given in `source_format`. It rejects all transaction filters and never sends Pinpoint's native filter language.
- **`summary`** is an explicit grouping. It returns at most 10 groups, puts failures and high latency first, and reports its own coverage. If groups were left out, the result is `partial` and includes an exact-items recovery action.
```json
{ "application_ref": "prod:checkout", "time": { "kind": "relative", "minutes": 5 },
"status": "failed", "output": "items" }
```
### `pinpoint_trace`
| Parameter | Type | Default | Notes |
|---|---|---|---|
| `view` | `summary` \| `events` | — | |
| `trace_ref` | string | — | A signed reference from a transaction search. It is bound to the caller and application. |
| `event_types` | (`error` \| `sql` \| `external` \| `rpc` \| `method`)[] ≤ 5 | — | For `events`. |
| `page_size` | int 1–50 | 20 | |
| `cursor` | string | — | |
- **`summary`** gives `elapsed_ms`, whether the source trace is complete, event counts by type, `slowest_calls`, `exceptions`, `sql_fingerprint` values together with their `timed_parent` method, and `external_calls`.
- **`events`** returns pages of normalized call-stack rows in source order. A filter that matches nothing returns an empty but complete page; it never falls back to unfiltered data.
- If Pinpoint returns a trace that belongs to another application, the call is denied (`PERMISSION_DENIED`). Redacted trace projections are cached for `MCP_TRACE_CACHE_TTL_SECONDS`.
### `pinpoint_topology`
| Parameter | Type | Default | Notes |
|---|---|---|---|
| `application_ref` | string | — | |
| `time` | time | — | Up to 30 minutes. |
| `focus` | `summary` \| `upstream` \| `downstream` \| `errors` \| `latency` | `summary` | |
| `top_k` | int 1–10 | 10 | |
| `cursor` | string | — | |
Returns one-hop dependency rows with `direction` (callers are upstream, callees are downstream, relative to the selected application), `dependency` (the remote endpoint), `total_count`, `error_count`, `slow_count`, `average_ms`, and `maximum_ms`. It always uses one hop with `bidirectional=false`. If Pinpoint reports an average above the maximum, the values are kept exactly as reported and a warning is added. It shows correlation, not proof of a root cause.
### `pinpoint_active_threads`
*Only registered when live diagnostics are enabled and available.*
| Parameter | Type | Notes |
|---|---|---|
| `application_ref` | string | |
Contacts live agents. If any agent times out or returns an invalid snapshot, coverage is `partial` with the reason `live_agent_unavailable`.
### `pinpoint_thread_dump`
*Only registered when live diagnostics are enabled and available.*
| Parameter | Type | Default | Notes |
|---|---|---|---|
| `application_ref` | string | — | |
| `agent_id` | string | — | Must belong to the application. |
| `kind` | `light` \| `full` | `light` | |
| `thread_name` | string | — | Optional filter. |
| `local_trace_id` | int ≥ 0 | — | Optional filter. |
| `limit` | int 1–10 | 10 | At most 10 threads, 20 stack frames each. |
Pinpoint Web needs `config.enable.activeThreadDump=true`. If Pinpoint answers HTTP 500, the tool returns `CAPABILITY_UNAVAILABLE` with the fix. Thread dumps are never used for readiness, preflight, or automatic retries, and each caller gets **1 per minute**.
---
## Response format
Every successful call returns this envelope, both as `structuredContent` and as JSON text:
```json
{
"status": "partial",
"request_id": "7c1b9f3e-2a44-4d7e-9a0b-1f2e3d4c5b6a",
"data": { "...": "tool-specific payload" },
"coverage": { "complete": false, "scanned": 8, "total": null, "reason": "page_limit" },
"next_cursor": "eyJraW5kIjoi...signature",
"next_actions": [
{ "tool": "pinpoint_trace", "reason": "Inspect the slowest failed transaction.", "arguments": { "view": "summary", "trace_ref": "..." } }
],
"warnings": ["..."]
}
```
| Field | Meaning |
|---|---|
| `status` | `ok` means complete. `partial` means bounded or incomplete evidence, so read `coverage` and `warnings`. `needs_input` means the server needs a decision (see `question` and `candidates`). |
| `coverage.complete` | `true` only when the whole requested population was covered. |
| `coverage.scanned` / `total` | Items examined, and the total when known. `total` is `null` when Pinpoint doesn't report one; it is never guessed. |
| `coverage.reason` | Why coverage is incomplete, for example `page_limit`, `no_observations`, or `live_agent_unavailable`. |
| `next_cursor` | Signed, expires after 10 minutes, and continues the same fixed query. |
| `next_actions` | At most 2 suggested follow-up calls with ready-to-use arguments. |
| `warnings` | Notes such as cache hits, excluded pings, or missing samples. |
| `question` / `candidates` | Present on `needs_input`. `candidates` holds up to 5 safe argument sets. |
---
## Error codes
A failed call returns `isError: true` with `{"error": {"code", "message"}}`. Messages never contain tokens, cursors, raw input, or upstream bodies.
| Code | When |
|---|---|
| `QUERY_TOO_BROAD` | Invalid or future window, conflicting parameters, or a request over a limit. It becomes `needs_input` when there is a safe candidate. |
| `UNSUPPORTED_FILTER` | The mode or filter combination isn't allowed, for example filters with scatter, a cursor with summary, or an unknown metric. |
| `PERMISSION_DENIED` | Missing authentication, or a trace or host group outside the caller's scope. |
| `CAPABILITY_UNAVAILABLE` | Optional module absent, application not in inventory, agent not in application, or thread dump rejected. |
| `UPSTREAM_TIMEOUT` | A Pinpoint request or the tool's deadline expired. |
| `UPSTREAM_UNAVAILABLE` | Network error, HTTP error, refused redirect, or retry budget used up. |
| `UPSTREAM_SCHEMA_CHANGED` | Pinpoint returned non-JSON (often a login page), malformed JSON, or an unexpected shape. |
| `EXPIRED_CURSOR` | The cursor or trace reference is expired, tampered with, or belongs to another caller or query. |
| `RATE_LIMITED` | The per-caller limit was hit, or Pinpoint returned HTTP 429. Don't retry in the same turn. |
| `RESULT_TOO_LARGE` | The upstream body or tool output is over its limit. Ask for fewer rows, points, event types, or threads. |
---
## Guardrails
### Read-only surface
- Only [allowlisted routes](#pinpoint-routes-used) can be built into URLs, and every request is a `GET`.
- Every tool is annotated read-only and non-destructive. The server has no mutation, admin, alarm, or config routes and no general HTTP passthrough.
- Pinpoint's native scatter filter language is never sent upstream.
### Input validation
- Strict schemas reject unknown properties, URLs used as application names, oversized lists, invalid or future windows, and unsafe page sizes.
- Decisions that matter (application, window, target, trace) are never guessed. A missing value returns `needs_input` before any upstream work happens.
### Time windows
| Tool | Maximum window |
|---|---|
| Health | 15 minutes (`current=true` means the last 5 minutes) |
| Transactions, topology | 30 minutes |
| Metrics, agent events | 24 hours |
### Result limits
- At most 5 applications per discovery page and 5 `needs_input` candidates.
- Exact transaction item pages are capped at 8. Scatter and summary default to 20 source rows, up to 50.
- Other row and event pages allow up to 50. Transaction summaries return at most 10 groups.
- Metrics return at most 120 points, 4 metric IDs, 2 agents, and 4 agent × metric pairs. A comparison is exactly 2 agents × 1 metric.
- Topology returns at most 10 one-hop rows per page. Thread dumps return 10 threads and 20 frames per thread.
- At most 2 `next_actions`.
### Rate limits (per caller, per minute, in memory)
| Class | Limit | Applies to |
|---|---:|---|
| all | 30 | Every call |
| expensive | 6 | Health, agent events, metric queries, transactions, traces, topology |
| live | 4 | `pinpoint_active_threads` |
| dump | 1 | `pinpoint_thread_dump` |
The limiter tracks at most 10,000 buckets, so its memory use is bounded.
### Concurrency, deadlines, and upstream protection
- At most `MCP_MAX_UPSTREAM_CONCURRENCY` (default 2, max 8) Pinpoint requests run at once across the whole process.
- Each tool call has a **20-second** deadline. Upstream requests time out after 10 s by default (trace routes after 15 s), and cancellation reaches the upstream request.
- Upstream requests refuse redirects (`redirect: error`), skip HTTP caching (`cache: no-store`), and accept only JSON responses.
- A network error or HTTP 502/503/504 is retried **once**, and only if the deadline leaves time.
- Upstream bodies are streamed and aborted once they pass 2 MiB (4 MiB for traces). Very large integers such as transaction IDs are kept as strings, so no precision is lost.
### Output ceiling
- Each result must fit in `MCP_MAX_OUTPUT_BYTES` (default 12 KiB). If it doesn't, the call **fails with `RESULT_TOO_LARGE`**. The server never trims a result silently. Services are built to fit their own limits and use cursors or recovery actions instead.
### Redaction (always on)
- **Removed fields:** arguments, parameters, SQL bind values, request and response bodies, headers, cookies, credentials, authorization, IPs, hostnames, ports, local and remote addresses, query objects, SQL metadata.
- **SQL** is replaced by a short SHA-256 fingerprint of the normalized statement, attached to its annotation row together with the timed parent method.
- **URLs and endpoints** keep only the path. Numeric and UUID segments become `{id}`, and query values become `[REDACTED]`.
- **Free text:** bearer tokens, `password=`/`secret=`/`token=`/`api_key=` values, and IPv4/IPv6 addresses are masked, and text is truncated. Stack traces are cut to 20 frames.
- Traces are redacted **before** they are cached. "Exact" modes never skip redaction.
### Cursors and ownership
- Cursors are HMAC-SHA256 signed with `MCP_CURSOR_HMAC_SECRET` and expire after **10 minutes**. Each one is bound to the caller, the application, a hash of the query, the resolved time window, and the position, so it can't be tampered with or reused for another query.
- Agent membership, trace ownership, and host-group access are rechecked at every relevant step.
### HTTP layer
- Only `/mcp` requires authentication. The health endpoints expose nothing sensitive.
- The `Host` header must be in `MCP_ALLOWED_HOSTS`. Requests with an `Origin` header are rejected unless the origin is in `MCP_ALLOWED_ORIGINS`.
- JSON request bodies are limited to 64 KB. Errors return generic bodies with no internal details.
### Privacy in logs
- Logs contain SHA-256 hashes of callers and applications, route templates, durations, status codes, byte counts, and cache and coverage outcomes. They **never** contain tokens, cursors, raw input, SQL, or trace bodies.
---
## Configuration
All configuration comes from environment variables, is checked once at startup, and fails closed. See [`configuration.example`](./configuration.example).
### Required
| Variable | Description |
|---|---|
| `PINPOINT_BASE_URL` | Pinpoint Web origin plus any context path. HTTP or HTTPS, with no credentials, query, or fragment. |
| `PINPOINT_SAFE_APPLICATION` | An existing application used for startup preflight probes. It does **not** limit which applications tools can query. |
| `MCP_PUBLIC_URL` | Canonical **HTTPS** URL whose path is exactly `/mcp`. It is also the JWT audience. |
| `MCP_JWT_PUBLIC_KEY_PEM` | Ed25519 SPKI public key in PEM format. Any other key type is rejected. |
| `MCP_CURSOR_HMAC_SECRET` | Random secret of at least 32 characters, different for each environment. |
### HTTP
| Variable | Default | Notes |
|---|---|---|
| `MCP_HTTP_HOST` | `0.0.0.0` | |
| `MCP_HTTP_PORT` | `3000` | 1–65535 |
| `MCP_ALLOWED_HOSTS` | hostname from `MCP_PUBLIC_URL` | Comma-separated hostnames with no scheme, port, path, or wildcard. |
| `MCP_ALLOWED_ORIGINS` | empty | Empty means any request with an `Origin` header is rejected. |
### Authentication
| Variable | Default | Range |
|---|---|---|
| `JWT_ISSUER` | `pinpoint-mcp-local` | ≤ 200 characters, exact match |
| `JWT_CLOCK_TOLERANCE_SECONDS` | `30` | 0–300 |
| `MCP_MAX_SESSION_SECONDS` | `31536000` | 60–31536000. The maximum token lifetime (`exp - iat`). |
### Pinpoint upstream and limits
| Variable | Default | Max |
|---|---:|---:|
| `PINPOINT_EXPECTED_VERSION` | `2.5.4` | — |
| `PINPOINT_REQUEST_TIMEOUT_MS` | `10000` | `60000` |
| `PINPOINT_TRACE_REQUEST_TIMEOUT_MS` | `15000` | `20000` |
| `MCP_MAX_UPSTREAM_CONCURRENCY` | `2` | `8` |
| `MCP_MAX_UPSTREAM_BODY_BYTES` | `2097152` | `16777216` |
| `MCP_MAX_TRACE_BODY_BYTES` | `4194304` | `16777216` |
| `MCP_MAX_OUTPUT_BYTES` | `12288` | `1048576` |
| `MCP_TRACE_CACHE_TTL_SECONDS` | `600` | `86400` |
| `NODE_EXTRA_CA_CERTS` | — | Path to a CA bundle if Pinpoint uses a private CA. |
### Optional modules
These default to **`false`** when unset. `configuration.example` and the Helm chart set them to `true`.
| Variable | Effect when `true` |
|---|---|
| `PINPOINT_URI_ENABLED` | Probe `/uriStat/summary` at startup and turn on URI metrics if it responds. |
| `PINPOINT_SYSTEM_METRICS_ENABLED` | Probe `/systemMetric/hostGroup` at startup and turn on system metrics if it responds. |
| `PINPOINT_LIVE_ENABLED` | Register `pinpoint_active_threads` and `pinpoint_thread_dump` if the client supports live counts and the safe application has at least one agent. |
Boolean values accept `true`, `false`, `1`, or `0`.
---
## Authentication
- **Tokens:** a JWT with `alg=EdDSA`, `iss` equal to `JWT_ISSUER`, `aud` equal to `MCP_PUBLIC_URL`, and `sub`, `iat`, and `exp`. `nbf` is enforced when present. `exp - iat` can't exceed `MCP_MAX_SESSION_SECONDS`.
- **Access:** any valid token gets the full read-only catalog across all applications, system host groups, and available live diagnostics. There are no per-application or per-tool scopes.
- **Key custody:** the server only verifies tokens, because it never has the private key. Mint tokens offline with `npm run mint-token`.
- **Revocation:** there is no revocation endpoint. To invalidate every token, create a new key pair, update `MCP_JWT_PUBLIC_KEY_PEM`, restart, and issue new tokens.
- **OAuth:** OAuth/OIDC discovery metadata isn't published. Clients that need an interactive OAuth login flow aren't supported yet. Any client that can set an `Authorization` header works.
---
## Pinpoint routes used
These are the only Pinpoint Web routes the server can call:
| Area | Routes |
|---|---|
| Core | `/applications`, `/serverTime` |
| Agents | `/agents/search-application`, `/getAgentInfo`, `/getAgentEvents` |
| Metrics | `/getApplicationStat/{chartType}/chart`, `/getAgentStat/{chartType}/chart`, `/getApplicationStat/dataSource/chart`, `/getApdexScore` |
| Transactions | `/getScatterData`, `/heatmap/drag` |
| Traces | `/transactionInfo`, `/traceViewerData` |
| Topology and health | `/getServerMapDataV2`, `/getResponseTimeHistogramDataV2` |
| URI module *(optional)* | `/uriStat/summary`, `/uriStat/chart` |
| System metrics *(optional)* | `/systemMetric/hostGroup`, `/systemMetric/hostGroup/host`, `.../collectedMetricInfoV2`, `.../collectedTags`, `.../collectedMetricData` |
| Live *(optional)* | `/agent/activeThread.pinpointws` (WebSocket), `/agent/activeThreadLightDump`, `/agent/activeThreadDump` |
`{chartType}` must be one of the metric types listed under [`pinpoint_metrics`](#pinpoint_metrics).
---
## Health, readiness, and logging
### Endpoints
| Endpoint | Auth | Behavior |
|---|---|---|
| `GET /health/live` | none | Always returns `200 {"status":"ok"}` while the process is up. |
| `GET /health/ready` | none | `200 {"status":"ready"}` after preflight passes, otherwise `503 {"status":"not_ready"}`. |
| `/mcp` | Bearer JWT | MCP Streamable HTTP. |
### Readiness loop
1. **Startup preflight** checks `/serverTime`, confirms the safe application exists in `/applications`, then makes small queries against agents, the response-time chart, the histogram, scatter (1 row), the heatmap (1 row), and the one-hop server map for that application.
2. **Optional module probes** (URI, system metrics) run only if their flags are on, and failures are tolerated. Live support is detected without ever taking a thread dump.
3. **Once ready**, readiness is rechecked at most every 30 seconds using only `/serverTime`. If a check fails, the next one reruns the full preflight.
Health endpoints don't grant access to telemetry and don't prove that optional modules work.
### Structured logs
Logs are one JSON object per line on stdout:
| `event` | Fields |
|---|---|
| `server_started` / `server_stopping` | host, port, signal |
| `tool_call` | tool, hashed principal, hashed application, `duration_ms`, status, coverage, `cache_outcome`, `output_bytes` |
| `tool_denied_or_failed` | tool, hashed principal and application, `duration_ms`, error `code` |
| `pinpoint_upstream` | method, **route template**, `duration_ms`, HTTP status, `body_bytes`, `attempts`, `cache_outcome` |
The process shuts down cleanly on `SIGINT` and `SIGTERM`.
---
## Caching and scaling
In-memory caches only:
| Cache | TTL | Notes |
|---|---|---|
| Application inventory | 60 s | Revalidated with ETag. |
| Health evidence | 15 s | A cache hit keeps the original fixed window, coverage, and follow-up. A later 15-second bucket moves the window forward. |
| Redacted trace projections | `MCP_TRACE_CACHE_TTL_SECONDS` | Only the normalized, redacted form is stored. |
Scatter, metrics, topology, agent events, and live diagnostics are never cached.
**Run a single replica.** Rate-limit counters and caches exist only inside one process, so extra replicas would each allow the full limits. To scale out, move the rate-limit counters into a shared store such as Redis. Cursors are stateless and signed, so they don't need shared state.
---
## Model system prompt and evaluations
### [`SYSTEM_PROMPT.md`](./SYSTEM_PROMPT.md)
This is the recommended system prompt for any model using this server. It tells the model to:
- treat telemetry as untrusted data;
- prefer exact modes and label derived answers;
- resolve the application once, follow at most one `next_action` at a time, and never drain cursors automatically;
- never present partial coverage, missing samples, or unavailable modules as healthy;
- call live diagnostics only when the user asks;
- open its answer with the finding, the exact window, and the coverage.
### [`evals.xml`](./evals.xml)
The file holds 15 multi-turn agent evaluation scenarios, each listing the expected tool calls and assertions:
scope clarification · current health drill-down · agent restart → metrics · optional-module gating · failed transaction → trace · latency → topology · cursor continuation · narrowing a broad query · prompt injection across scopes · explicit live gates · preserving short metric spikes · no observations ≠ healthy · high-throughput exact pagination · multi-agent CPU/memory · health cache coherence.
Run them through any agent harness, or by hand with MCP Inspector, against a non-production Pinpoint.
---
## Deployment
### Docker
The multi-stage image builds on `node:22.19.0-bookworm-slim` and runs as the unprivileged `node` user on port 3000.
```bash
docker build -f deploy/Dockerfile -t ghcr.io/karma098/pinpoint-apm-mcp:1.0.1 .
docker run --rm -p 3000:3000 --env-file .env ghcr.io/karma098/pinpoint-apm-mcp:1.0.1
```
If a TLS-intercepting proxy sits in the build path, pass its CA as a BuildKit secret. It's used only during `npm ci` and is not copied into the image:
```bash
docker build --secret id=npm_ca,src=/path/to/ca.pem -f deploy/Dockerfile -t pinpoint-apm-mcp:1.0.1 .
```
### Kubernetes (Helm)
The chart is at `deploy/helm/pinpoint-mcp`.
**What it deploys:**
- A Deployment with **1 replica** and rolling updates (`maxUnavailable: 0`, `maxSurge: 1`). Readiness and liveness probes hit `/health/ready` and `/health/live`.
- Pod hardening: `runAsNonRoot` (UID/GID 1000), `readOnlyRootFilesystem`, `allowPrivilegeEscalation: false`, all capabilities dropped, `RuntimeDefault` seccomp, `automountServiceAccountToken: false`, and a 32 Mi `emptyDir` at `/tmp`.
- A ClusterIP Service, a ConfigMap (a config checksum triggers restarts), a dedicated ServiceAccount, and an optional Ingress.
- A NetworkPolicy: ingress only on the MCP port (optionally limited to namespace, pod, or CIDR selectors). Egress only to cluster DNS and the Pinpoint CIDRs and port.
- An optional private CA mounted from a Secret, which sets `NODE_EXTRA_CA_CERTS`.
**It never creates Secrets.** Create them first:
```bash
kubectl create namespace mcp
kubectl -n mcp create secret generic pinpoint-mcp-secrets \
--from-literal=MCP_CURSOR_HMAC_SECRET="$(openssl rand -base64 48)" \
--from-file=MCP_JWT_PUBLIC_KEY_PEM=jwt-public.pem
# optional, for a Pinpoint private CA
kubectl -n mcp create secret generic pinpoint-ca --from-file=ca.crt=/path/to/ca.crt
```
**Values file.** `*-values.yaml` is gitignored.
```yaml
# my-values.yaml
image:
repository: ghcr.io/karma098/pinpoint-apm-mcp
tag: "1.0.1" # mutable tags such as latest/main/dev are rejected by the schema
pinpoint:
baseUrl: http://pinpoint-web.pinpoint.svc:8080
safeApplication: my-safe-app
# caSecretName: pinpoint-ca
mcp:
publicUrl: https://pinpoint-mcp.example.com/mcp
allowedHosts: [pinpoint-mcp.example.com]
allowedOrigins: []
jwt:
issuer: pinpoint-mcp-local
existingSecret: pinpoint-mcp-secrets
features:
uriMetrics: true
systemMetrics: true
live: true
networkPolicy:
pinpointCidrs: [10.0.0.10/32] # required: NetworkPolicy cannot match DNS names
pinpointPort: 8080
# ingressNamespaceSelector: { kubernetes.io/metadata.name: ingress-nginx }
ingress:
enabled: true
className: nginx
host: pinpoint-mcp.example.com
tls:
secretName: pinpoint-mcp-tls
```
Rendering **fails on purpose** until `pinpoint.baseUrl`, `pinpoint.safeApplication`, an HTTPS `mcp.publicUrl` ending in `/mcp`, `mcp.allowedHosts`, `existingSecret`, and `networkPolicy.pinpointCidrs` are set. `ingress.host` is also required when ingress is enabled.
```bash
helm lint deploy/helm/pinpoint-mcp -f my-values.yaml
helm template pinpoint-mcp deploy/helm/pinpoint-mcp -n mcp -f my-values.yaml > rendered.yaml
kubectl apply --dry-run=server -f rendered.yaml
helm upgrade --install pinpoint-mcp deploy/helm/pinpoint-mcp -n mcp -f my-values.yaml
kubectl -n mcp rollout status deployment/pinpoint-mcp-pinpoint-mcp
```
If you disable ingress, an existing TLS gateway must serve `mcp.publicUrl` and route `/mcp` to the Service. Configure the ingress to reject or redirect plaintext HTTP. Restart the Deployment after rotating the cursor secret, JWT key, or CA.
**Rollback and emergency stop.** Neither affects Pinpoint.
```bash
helm rollback pinpoint-mcp -n mcp
kubectl -n mcp scale deployment/pinpoint-mcp-pinpoint-mcp --replicas=0
```
---
## Production rollout checklist
1. Confirm the Pinpoint context path, network reachability, TLS chain, version, retention, enabled modules, and a safe application.
2. Run `npm run check`, `helm lint`, and a server-side dry run.
3. Deploy and check `/health/live` and `/health/ready`.
4. With Inspector, list tools and call `pinpoint_discover` with `view=capabilities`. Query the safe application, then another application.
5. Check that it fails closed: expired or tampered cursors, wrong JWT issuer or audience, missing modules, and oversized requests.
6. Watch Pinpoint latency and CPU headroom, along with MCP timeouts, rate denials, output sizes, and partial results.
7. Try a light thread dump by hand on a non-critical agent before relying on live tools.
8. Load-test gradually against the safe application, and only with approval.
---
## Troubleshooting
| Symptom | Likely cause and fix |
|---|---|
| Startup error `... is required` / `must use https` / `must be a valid Ed25519 SPKI public key` | Configuration check failed. Fix the named variable. |
| `/health/ready` stays `503` | Preflight failed. Check that `PINPOINT_BASE_URL` is reachable, that `PINPOINT_SAFE_APPLICATION` exists exactly as named in Pinpoint, and look at the `pinpoint_upstream` logs. |
| `401` / `invalid_token` | Token `iss` or `aud` doesn't match `JWT_ISSUER` / `MCP_PUBLIC_URL` exactly, the token has expired, its lifetime is over `MCP_MAX_SESSION_SECONDS`, or it was signed with a different key. |
| Request rejected before auth | The `Host` header isn't in `MCP_ALLOWED_HOSTS`, or the client sent an `Origin` that isn't in `MCP_ALLOWED_ORIGINS`. |
| `UPSTREAM_SCHEMA_CHANGED: non-JSON response` | Pinpoint returned HTML, usually a login or SSO page. Make Pinpoint Web reachable from the server without an interactive login. |
| Live tools are missing | `PINPOINT_LIVE_ENABLED` isn't `true` (it defaults to `false`), the safe application has no agents, or the client listed tools before readiness. Reconnect after `/health/ready` returns 200. |
| URI or system metrics return `CAPABILITY_UNAVAILABLE` | The flag is off or the startup probe failed. The module may not be installed in Pinpoint. |
| Thread dump returns `CAPABILITY_UNAVAILABLE` | Set `config.enable.activeThreadDump=true` on Pinpoint Web and make sure the agent is connected. |
| `RESULT_TOO_LARGE` | Lower `page_size`, `max_points`, `event_types`, or `limit`, or follow `next_cursor`. |
| `EXPIRED_CURSOR` | Cursors last 10 minutes and are bound to one caller and one query. Start the query again. |
| `RATE_LIMITED` | Wait for the next minute. Calls that don't touch telemetry (discovery, catalogs) are cheaper. |
---
## Limitations
- Built for **Pinpoint Web 2.5.4**. Other versions may return different shapes.
- **Single replica only**, because rate limits and caches live inside one process.
- **Static tokens:** no per-application or per-tool scopes, no online revocation, no OAuth discovery.
- Read-only: no alarms, admin, config, or write operations, by design.
- The code includes a 12-tool "granular" profile, but it is internal only and the server doesn't expose it.
---
## Development
```bash
npm ci
npm run dev # watch mode (tsx)
npm run build # strict tsc → dist/
npm test # node:test suite via tsx
npm run check # build + test
```
Tests live in `tests/`:
| File | Covers |
|---|---|
| `security.test.ts` | Config fails closed, JWT verification (EdDSA, issuer, audience, expiry), agent ownership checks, public health endpoints vs protected `/mcp`, Host/Origin validation |
| `mcp.test.ts` | Workflow and granular catalogs, live-tool gating, fresh Streamable HTTP clients, `needs_input` before telemetry, cursor-only continuation, discovery paging |
| `runtime.test.ts` | Application policy matching, signed query-bound cursors, fail-closed output ceiling, strict schema rejection |
| `transactions.test.ts` | Exact scatter without native filters, cursor continuation, 8-item default and smaller candidate, grouped coverage and recovery |
| `metric-projection.test.ts` | Point budgets, `-1` exclusion, extrema timestamps and agent attribution, empty-chart semantics, unknown shapes fail closed |
| `debugging.test.ts` | Trace summaries and filters, health histogram + Apdex and caching, agent events after restart, active-thread partial coverage, metric series, memory alias, comparisons, topology |
| `pinpoint/adapter.test.ts` | Route allowlist, same-origin URL building, ping exclusion, normalizers, redaction, thread-dump caps, malformed fixtures |
| `pinpoint/client.test.ts` | Apdex and thread-dump requests, disabled-dump errors, ETag revalidation, single retry on 502/503/504, trace deadline, no credential forwarding, active-thread WebSocket |
---
## Contributing
Issues and pull requests are welcome.
1. Fork the repository and create a branch.
2. Keep changes focused, and add a regression test next to the code you change.
3. Run `npm run check` before opening a pull request.
4. Describe what changed and why. For new Pinpoint routes, link the Pinpoint source that defines the route's contract.
Please **never commit** real Pinpoint hostnames, application names, IPs, tokens, keys, or captured telemetry. Use sanitized fixtures. New upstream routes must stay read-only and be added to the allowlist in `src/pinpoint/routes.ts`.
---
## Security
If you find a vulnerability, please **don't open a public issue**. Report it privately through GitHub's private vulnerability reporting (**Security → Report a vulnerability**) so it can be fixed before it's disclosed.
Operator reminders:
- Keep the JWT private key offline, and never put it in the server, a ConfigMap, or Git.
- Generate a separate `MCP_CURSOR_HMAC_SECRET` for each environment.
- Always serve `/mcp` over TLS, and don't expose MCP Inspector to untrusted networks.
---
## License
Licensed under the [Apache License 2.0](./LICENSE).
## Acknowledgements
- [Pinpoint APM](https://github.com/pinpoint-apm/pinpoint)
- [Model Context Protocol](https://modelcontextprotocol.io) and its TypeScript SDK
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessNo issues