Vantage Telemetry MCP Server

A Model Context Protocol (MCP) server that captures and exposes usage, cost, productivity, and activity metrics for local-first software engineering teams.

It functions as the telemetry agent on a developer's machine, running in the background while plugged into an MCP-capable client (like VS Code/Cline, Cursor, Claude Code, or Windsurf).

Setup

One command, and it works whether your team's harness/telemetry service run on localhost, on a machine on your network, or in the cloud.

1. Prerequisites

2. Point it at your telemetry service

Local (default) — nothing to configure. If you're running the stack locally via Docker, skip straight to step 3; setup.py defaults to http://localhost:50223 (harness) and http://localhost:50224 (telemetry).

Remote or cloud-hosted service — pass the real URLs once, either as flags or environment variables:

python setup.py \
  --harness-url https://harness.yourcompany.com \
  --telemetry-url https://telemetry.yourcompany.com

$env:HARNESS_URL   = "https://harness.yourcompany.com"
$env:TELEMETRY_URL = "https://telemetry.yourcompany.com"
python setup.py

HTTPS works automatically (Python's standard TLS verification applies — see Troubleshooting if your service uses a self-signed cert). You only need to do this once per machine: the URLs are saved to ~/.vantage/config.json, and every future python setup.py re-run reuses them without the flags.

3. Get registered — identity, team, and access token

setup.py will ask for (or auto-resolve) three things and will not proceed with a blank team ID — every developer's telemetry must be attributable to a real team, not silently dumped into "unknown":

• User ID — auto-detected from git config user.email. If that's not set, it prompts (never silently substitutes your OS username without asking).

• Team ID — pass --team-id engineering, set VANTAGE_TEAM_ID, or answer the prompt. Required.

• API token, scoped to that user + team. Two ways a remote/shared deployment typically hands these out — ask whoever runs your harness which applies to you:

  1. Self-service with a shared bootstrap token. Your admin gives the team a single admin token once; each developer runs setup with it and gets their own personal token minted automatically:

     python setup.py --admin-token <the-team-bootstrap-token>
     # or: export VANTAGE_ADMIN_TOKEN=<the-team-bootstrap-token>

  2. Admin issues your token directly. They run POST {harness_url}/admin/tokens for you and send you the result; paste it in when setup.py prompts for a token, or supply it upfront with --token <your-personal-token> (this skips token creation entirely — don't confuse it with --admin-token, which is only for minting a new token and expects an admin-scoped credential, not a personal one).

The resolved token is cached in ~/.vantage/config.json and re-validated (not re-created) on every future run, so this is also a one-time step per machine.

4. Run it

python setup.py

This single run also: installs Python dependencies (mcp, watchdog, psutil, httpx), detects which AI tools are actually installed on your machine (Cursor, Claude Code, Codex, Kiro, VS Code+Cline, VS Code+Copilot — nothing is written for tools you don't have), registers this MCP server with each of them, installs the Claude Code Stop hook and Codex notify wrapper, and drains any telemetry that was queued locally before setup ran. It's safe to re-run any time — every config file is merged, not overwritten, and a second run only updates what actually changed.

Non-interactive (CI, scripted/fleet installs):

python setup.py --user-id you@company.com --team-id engineering \
  --harness-url https://harness.yourcompany.com \
  --telemetry-url https://telemetry.yourcompany.com \
  --admin-token "$VANTAGE_ADMIN_TOKEN" --non-interactive

--dry-run resolves and prints identity + detected tools without writing anything. See python setup.py --help for every flag.

5. Reload your IDE(s)

6. Confirm telemetry is actually reaching the remote service

Setup succeeding doesn't by itself prove events are landing on a remote service — network paths that work for the setup script's one-time API calls aren't necessarily open for the MCP server's own background delivery. Trigger a bit of real activity (send one prompt, edit a file, etc.), then check any of these, in order of convenience:

1. Ask the MCP directly. From your AI tool's chat, invoke the get_collection_health tool. It reports whether a harness is configured, how many events are still pending delivery, and the oldest pending event's timestamp — if Pending delivery events stays at 0 (or keeps draining to 0), delivery is working.

2. Watch the delivery log. ~/.vantage/telemetry_delivery.log appends one line per attempt: OK ... means delivered, TIMEOUT ... / CONN ... mean the remote service wasn't reachable (network/firewall — see below), FAIL400 ... means the service rejected the payload (schema/auth issue).

3. Watch the outbox drain. ~/.vantage/telemetry_outbox.jsonl holds events not yet acknowledged. It should trend toward empty. A file that only grows means events aren't reaching the remote service at all.

4. Check the dashboard on the remote service itself: {telemetry_url}/dashboard — filter by your user_id and you should see your own activity appear within roughly 15–30 seconds of it happening (matching the watchers' default poll intervals).

7. What happens if the remote service isn't reachable yet

setup.py deliberately does not fail if it can't reach the harness or telemetry service at setup time (VPN not connected yet, service mid-deploy, etc.) — it still writes every IDE registration and finishes normally. Events generated in the meantime queue in ~/.vantage/telemetry_outbox.jsonl and are delivered automatically the moment connectivity comes up: the background worker retries with exponential backoff, and a periodic reconcile pass (every 5 minutes) re-sends anything still stuck. No re-run of setup.py, IDE reload, or any other developer action is needed once the network path opens up — this is the same self-healing delivery pipeline described below, and it's what makes "telemetry starts recording as soon as setup finishes" true even under imperfect first-run network conditions.

Troubleshooting

The vantage-client repo's setup.ps1 still exists for its own Continue-config and Cursor-BYOK responsibilities, but setup.py here is now the canonical way to register this MCP server itself — no second repo required.

Related MCP server: agent-observability-mcp

🚀 Key Features

• Usage Analytics: Tracks prompts, assistant response lengths, and token usage.

• Cost Analytics: Integrates with Vantage Harness to display estimated cloud LLM costs vs. actual costs and net local-routing savings.

• Productivity Metrics: Monitors active typing session intervals, file additions/deletions, and code lines added/deleted in real-time.

• Audit Logs: Maintains a secure, local developer activity trail (file edits, command runs, git actions).

• Self-Bootstrapping: Automatically installs required Python packages (mcp, watchdog, psutil, httpx) on first startup.

Telemetry event contract and privacy

Events use an additive versioned envelope (schemaVersion, eventId, occurredAt, deviceId, sessionId, workspaceId, collectionMethod) while retaining the legacy flat fields consumed by Vantage Harness. Secrets and machine-specific absolute paths are filtered before local persistence or network delivery. Failed deliveries remain in ~/.vantage/telemetry_outbox.jsonl and retry with bounded exponential backoff.

Filesystem events deliberately use editOrigin: unknown; only an IDE-specific source such as Cursor's scored-commit database may claim AI/human attribution, and every attribution includes its source and confidence.

Additional capture sources: Grok Build and GitHub Copilot Chat

Neither Grok Build (xAI's CLI agent) nor GitHub Copilot Chat (VS Code) connects to this server as an MCP host, so — like Cursor — they're captured by polling the files/database each tool already maintains, not by a hook or notify callback:

• GrokSessionWatcher polls ~/.grok/sessions/<cwd>/<uuid>/ every 30s (VANTAGE_GROK_POLL_INTERVAL_SEC) for new or updated sessions and emits a usage event from signals.json's contextTokensUsed. Grok exposes no prompt/completion split, so the context footprint is attributed entirely to input tokens (tokenEstimateSource: grok_context_footprint) — a lower bound, not an exact count.

• CopilotChatWatcher polls VS Code's github.copilot-chat/session-store.db every 30s (VANTAGE_COPILOT_POLL_INTERVAL_SEC) for new turns and emits an agent_turn event (files touched, tools invoked, message lengths only — never content). Copilot exposes no token/cost data locally at all; usage is metered by GitHub server-side as "premium requests," visible only through GitHub's own usage API. Treat Copilot's local signal as activity, not spend.

• KiroLogWatcher tails q-client.log under every Kiro launch's log directory (~/AppData/Roaming/Kiro/logs/<timestamp>/window*/exthost/ kiro.kiroAgent/) every 30s (VANTAGE_KIRO_POLL_INTERVAL_SEC), using a byte-offset checkpoint per file like claude_code_stop_hook.py. Verified against real Kiro request logs: Kiro's agent panel calls AWS's CodeWhisperer runtime, and the actual streamed response is logged as an empty {} (the SDK can't serialize an event stream) — no token or cost figure ever reaches this file. Capture is therefore activity-tier only (conversation id, request success/failure, conversation depth, requested model — often literally "auto", not a resolved model name), emitted as agent_turn, the same tier as Copilot's signal.

All three watchers use the same singleton-lock + checkpointed-state pattern as CursorDbWatcher, so multiple MCP instances never double-report.

Claude Code: push + pull, not push OR pull

Claude Code's primary capture path is the Stop hook (hooks/claude_code_stop_hook.py) — fast and real-time, but entirely dependent on ~/.claude/settings.json's hooks.Stop config being valid and actually invoked. That dependency is a real, observed failure mode: a malformed settings.json once went unnoticed for ~17.5 hours with zero indication anything was wrong, because nothing independent of the hook was watching.

ClaudeTranscriptWatcher closes that gap by polling ~/.claude/projects/*/*.jsonl every 5 minutes (VANTAGE_CLAUDE_RECONCILE_INTERVAL_SEC) and re-scanning any transcript that hasn't been written to in the last 60 seconds (VANTAGE_CLAUDE_RECONCILE_DEBOUNCE_SEC) — the debounce exists purely so it never races a live Stop hook for the same bytes. It shares the exact same checkpoint file and token-accounting logic as the hook (claude_transcript.py), so whichever mechanism sees a given range of transcript bytes first claims it: the hook stays the fast path when it's working, and the watcher silently fills in whatever it missed when it isn't. Neither can double-count the other's work, and neither can permanently miss usage the other one dropped.

This mirrors how Grok/Copilot/Kiro are captured — periodic reconciliation against the tool's own on-disk source of truth — applied as a second, independent layer behind Claude Code's existing hook rather than a replacement for it.

Codex: the pull watcher IS the primary source, not a safety net

Codex is different from Claude Code in one important way: its notify hook (codex_notify_wrapper.py) carries no token counts at all — only turn-completion metadata (turn id, cwd, assistant message length). There has never been a push-based source of Codex usage/cost data in this repo.

CodexRolloutWatcher polls ~/.codex/sessions/<Y>/<M>/<D>/rollout-*.jsonl every 30s (VANTAGE_CODEX_POLL_INTERVAL_SEC) for token_count events — Codex's own precomputed per-turn token delta (payload.info.last_token_usage), far more precise than anything the notify hook exposes. Each token_count event becomes its own usage event (real per-turn granularity, not an aggregated backfill like Claude's), using the same checkpointed-offset pattern (codex_transcript.py) keyed per rollout file. Since Codex already registers vantage-telemetry-mcp as its own MCP server, this watcher runs automatically whenever Codex is used — no "only captured while some other tool is open" caveat like Grok has.

One practical note: on first run, checkpoints start at offset 0 for every existing rollout file, so the initial poll backfills full historical usage across every past session — this can mean a burst of individual usage events (one per turn, potentially hundreds across months of history) sent as sequential HTTP POSTs. This is a one-time cost per rollout file; steady state after that is just the handful of new turns since the last poll.

codex_notify_wrapper.py's agent_turn event is unchanged and still useful as a fast "a turn just completed" signal — it's just no longer the only thing Codex reports, and was never the source of token/cost numbers.

🛠️ Architecture

       VS Code / Cursor / Cline / Claude Code / Codex / Kiro
                         |
                         v (stdio / JSON-RPC)
               +-------------------+
               |   Telemetry MCP   | <---+ Watches Workspace File Changes,
               +-------------------+       Git Branch & Commit logs, and
                 /               \         each tool's own session files
                /                 \
               v                   v
      Local JSONL File    Telemetry Service  (local, LAN, or cloud)
  (~/.vantage/telemetry.jsonl)   {telemetry_url}/v1/telemetry

{telemetry_url} is whatever you configured in Setup — the same delivery code path runs unchanged whether that's localhost, a machine on your LAN, or a cloud-hosted service.

⚙️ Manual IDE configuration (fallback reference)

python setup.py (see Setup) does all of this automatically — use this section only if setup.py can't run on your platform, or you need to hand-verify/adjust one tool's registration. VANTAGE_WORKSPACE_DIR is normally left unset on purpose: mcp_server.py falls back to the process's own working directory, so the same registration correctly attributes telemetry no matter which project the IDE happens to launch it from. Only set it if you specifically want to pin one registration to a single fixed project.

1. VS Code (Cline)

{
  "mcpServers": {
    "vantage-telemetry": {
      "command": "python",
      "args": ["d:/root/projects/vantage-telemetry-mcp/server.py"],
      "disabled": false
    }
  }
}

Path: %APPDATA%\Code\User\globalStorage\saoudrizwan.claude-dev\settings\cline_mcp_settings.json

2. Cursor

~/.cursor/mcp.json:

{
  "mcpServers": {
    "vantage-telemetry": {
      "command": "python",
      "args": ["d:/root/projects/vantage-telemetry-mcp/server.py"]
    }
  }
}

3. Claude Code (CLI)

~/.claude.json:

{
  "mcpServers": {
    "vantage-telemetry": {
      "command": "python",
      "args": ["d:/root/projects/vantage-telemetry-mcp/server.py"]
    }
  }
}

Plus the Stop hook in ~/.claude/settings.json — see Claude Code: push + pull above for the exact schema; do not flatten hooks.Stop entries to {"type": "command", ...} without the hooks: [...] wrapper, that shape is invalid and silently breaks the whole Stop hook array.

🛠️ Exposed MCP Tools

The telemetry server registers several tools that can be invoked by AI agents or developers to log events or request local metrics:

1. Logging Tools

• log_usage_analytics(model, prompt_tokens, completion_tokens, duration_ms): Record custom LLM invocation usage.

• log_cost_analytics(estimated_cost_usd, actual_cost_usd, savings_usd, route, model): Log AI routing financial savings.

• log_productivity_metrics(task_id, status, lines_added, lines_deleted, details): Track progress of specific development task IDs.

• log_audit_log(action, target, status, details): Manually write to the developer activity trail.

2. Querying & Reporting Tools

• get_telemetry_summary(): Returns a comprehensive markdown dashboard showing total logs, LLM token spending, local routing savings rates, coding metrics (lines written vs deleted), and recent activities.

• get_productivity_report(period): Get detailed coding speeds and list of touched files for 'today', 'week', or 'all'.

• get_cost_report(): Returns an overview of total LLM calls, local-handling ratio, estimated baseline cloud spend, actual spend, and net savings.

• get_audit_trail(limit): Displays a list of recent files modified, Git operations, and active sessions.

📁 File Locations

• Shared Config: ~/.vantage/config.json (harness/telemetry URLs, user ID, team ID, API key — written by setup.py).

• Local Telemetry Log: ~/.vantage/telemetry.jsonl (durable, append-only record of every event ever captured, regardless of delivery status).

• Outbox: ~/.vantage/telemetry_outbox.jsonl (events not yet acknowledged by the telemetry service — see Setup step 6).

• Delivery log: ~/.vantage/telemetry_delivery.log (one line per delivery attempt: OK/TIMEOUT/CONN/FAIL400/DEAD/DUP/SKIP).

• Dead letter: ~/.vantage/telemetry_dead_letter.jsonl (events the service permanently rejected — a 400 response — so they stop retrying).

• Delivered-id ledger: ~/.vantage/telemetry_delivered_ids.json (dedup ledger so the periodic reconcile pass never re-sends something already acknowledged).

• Per-tool watcher state: cursor_watcher_state.json, grok_watcher_state.json, copilot_watcher_state.json, kiro_watcher_state.json, claude_stop_offsets/<session>.json, codex_rollout_offsets/<rollout>.json — poll cursors / byte-offset checkpoints, one family per capture source (see the per-tool sections above).

Composer telemetry spam

Cursor multi-root workspaces start one MCP server per folder, which used to duplicate cursor_usage POSTs to the harness. The watcher now uses a process lock (~/.vantage/cursor_db_watcher.lock) so only one instance polls Cursor’s ai_code_hashes table, aggregates hashes by requestId, and debounces (default 60s, VANTAGE_CURSOR_USAGE_INTERVAL_SEC).

After upgrading this repo, run Developer: Reload Window in Cursor so the MCP process reloads. New events include hashCount and a details summary; bare fileName-only lines indicate a stale MCP process still running old code.