Skip to main content
Glama
debaditya-mohankudo

Splunk Intelligence MCP Server

Splunk Intelligence

A local Splunk investigation stack that ingests exports (JSON/CSV) or runs live SPL queries, applies deterministic detectors, and drives a structured multi-iteration investigation loop via MCP tools exposed to AI agents (GitHub Copilot or Claude Code). Everything runs on-device — no data leaves the machine.

How it works

Splunk export (JSON/CSV)  ──or──  Splunk REST API
    └─> parsers.py        # Polars DataFrame: field extraction, timestamp normalisation
    └─> detectors.py      # rule-based: spikes, patterns, cert anomalies, correlations,
    │                     #   severity, host rankings, slow queries, numeric anomalies
    └─> connector.py      # facade: loading, detection, run state — no HTTP, no server
    │                     #   process; MCP tools, the TUI, runner.py, and its own CLI
    │                     #   all call into it directly
    └─> mcp_server.py     # FastMCP: exposes investigation tools to Copilot / Claude
    └─> tui.py            # terminal UI: run history + live progress, reads splunk.db directly
    └─> runner.py         # CLI orchestrator
    └─> reports/          # generated markdown reports
    └─> logs/             # per-run JSONL structured logs (audit trail — every
    │                     #   investigate/pause/hint/done action, not just the CLI pipeline)
    └─> splunk.db         # SQLite: events, findings, reports, queries, active_runs, alerts per run_id

standalone/               # top-level dir: processes that run outside the MCP/agent loop
    └─> agent.py          # optional: standalone LangGraph ReAct loop (--investigate flag),
    │                     #   via splunk/llm_backends.py — ollama / claude_cli / copilot_cli
    └─> watcher.py        # standalone process (python -m standalone.watcher) — polls Splunk on an
                          #   interval, runs detectors, writes hits to splunk.db's alerts table;
                          #   consumed via splunk__check_alerts / splunk__ack_alert

The investigation loop is self-contained — splunk__submit_report returns {status, findings, next} and the agent loops on its own without external hooks.

Copilot/Claude via MCP is the primary reasoning path — no Ollama or CLI subprocess required. For environments without either, standalone/agent.py provides an optional standalone LangGraph ReAct agent, enabled via uv run python -m splunk --input <file> --investigate (requires uv sync --extra llm). See Standalone agent below for backend options.

Related MCP server: Splunk MCP for SOC Operations

Quick start

1. Install prerequisites

  • Python 3.12+

  • uvbrew install uv

  • Splunk instance URL (set SPLUNK_URL env var; required for live queries only)

2. Install dependencies

uv sync --extra dev
uv run playwright install chromium

3. Configure Splunk URL (live queries only)

echo "SPLUNK_URL=https://your-splunk-instance:8089" > .env

4. Authenticate to Splunk (live queries only)

uv run python -m splunk.auth

This opens a visible Chromium window via Playwright. Complete the SSO login manually. The session cookie is saved to ~/.splunk/auth.json and loaded automatically on every live query. Re-run when your session expires (Splunk uses SSO/SAML — password login is not available).

5. Run an investigation

# From a local export file
uv run python -m splunk --input results/cert_errors.json

# Live SPL query
uv run python -m splunk --live --spl "index=pki sourcetype=ocsp_error" --earliest -6h

Via AI agent (MCP tools)

No server process required — start the MCP tool server, and optionally the TUI:

# Terminal 1 — MCP tool server
uv run python -m splunk.mcp_server

# Terminal 2 (optional) — terminal UI for watching live investigation progress
uv run python -m splunk.tui

Then ask Copilot or Claude: "Start a Splunk investigation on results/cert_errors.json"

The agent calls splunk__investigate_start, reasons over findings, and loops via splunk__submit_report until confident. See AGENTS.md for the full loop protocol.

The TUI reads splunk.db directly for run history and the rendered report, and polls the active_runs table for live iteration/confidence/event-count every ~2s — no HTTP involved. Because every connector function writes to active_runs regardless of which process calls it, the TUI shows live per-iteration progress for both MCP/Claude-driven investigations and the standalone --investigate agent path — previously (before this design), MCP-driven progress was invisible to any other process since it only lived in an in-memory dict inside whichever process was running it.

No MCP client available? Use the connector CLI

Same investigation engine, no MCP tool-calling required:

uv run python -m splunk.connector start --source results/cert_errors.json
uv run python -m splunk.connector submit-report --run-id <id> --report "..." --queries "-- area\nindex=pki ..."
uv run python -m splunk.connector get-findings --run-id <id>
uv run python -m splunk.connector pause --run-id <id>
uv run python -m splunk.connector hint --run-id <id> --text "focus on web-01 after 14:30 UTC"

Standalone agent (--investigate)

For environments without Copilot or Claude Code driving MCP tools directly, standalone/agent.py runs its own LangGraph ReAct loop over the same detector findings and produces the same kind of markdown report. It's a fallback, not the primary path — prefer the MCP flow above when available.

uv sync --extra llm   # pulls in langgraph, langchain-core, langchain-ollama

uv run python -m splunk --input results/cert_errors.json --investigate

The chat backend driving the loop is selected via SPLUNK_AGENT_BACKEND (default ollama):

Backend

Requires

Notes

ollama (default)

ollama serve running locally + a pulled model

Model via SPLUNK_LLM_MODEL (default qwen2.5:14b)

claude_cli

claude on PATH, already logged in to Claude Code

No API key needed — reuses your existing login. Model via SPLUNK_CLAUDE_CLI_MODEL (default sonnet)

copilot_cli

copilot on PATH, already logged in

No API key needed. Model via SPLUNK_COPILOT_CLI_MODEL (default claude-sonnet-4.5)

# Ollama (default) — needs `ollama serve` running and the model pulled
ollama pull qwen2.5:14b
uv run python -m splunk --input results/cert_errors.json --investigate

# Claude CLI — no separate server process, reuses your `claude` login
SPLUNK_AGENT_BACKEND=claude_cli \
  uv run python -m splunk --input results/cert_errors.json --investigate

# Copilot CLI
SPLUNK_AGENT_BACKEND=copilot_cli \
  uv run python -m splunk --input results/cert_errors.json --investigate

claude_cli/copilot_cli shell out to the CLI non-interactively (claude -p / copilot -p) with the CLI's own tool use disabled, bridging tool-calling by hand via a small JSON protocol — see splunk/llm_backends.py and splunk/cli_tool_protocol.py for how. One CLI session is opened per investigation and reused (--resume) across all ReAct iterations rather than starting cold every turn.

SPLUNK_AGENT_MAX_ITER (default 10) caps ReAct loop iterations regardless of backend.

Claude Code skills

  • /splunk-investigate <input> — the investigation loop (splunk__investigate_start → reason over findings → splunk__submit_report → repeat). <input> is a file path or an SPL query — one skill handles both: /splunk-investigate results/cert_errors.json or /splunk-investigate "index=pki sourcetype=ocsp_error" --earliest -6h If invoked with no argument, it asks.

MCP Tools

Tool

Purpose

splunk__investigate_start

Load file or live SPL query, run detectors, return structured findings + run_id

splunk__submit_report

Submit a markdown report and follow-up SPL queries; returns {status, findings, next} (continue) or {status, ui_url} (done) — either may also carry advisory repo_path_nudge/confidence_nudge/followup_nudge keys, never blocking, just surfacing something worth noting in the final summary

splunk__get_findings

Read current findings for an active run without advancing the loop

splunk__pause

Stop the loop after the current iteration

splunk__hint

Inject an analyst hint that shapes the next iteration

splunk__query_examples

Return past SPL queries from splunk.db to ground follow-up queries

splunk__lsp_call_chain

Trace a function/symbol through a microservice's call graph to find which code path produced a log error (requires repo_path)

splunk__check_alerts

Read unacknowledged alerts written by the standalone watcher (standalone/watcher.py)

splunk__ack_alert

Mark a watcher alert as acknowledged so it stops appearing in splunk__check_alerts

Onboarding (new team members)

An interactive onboarding prompt is available for GitHub Copilot. In VS Code Copilot Chat, attach .github/prompts/onboard.prompt.md via the # file picker — Copilot will walk you through setup, auth, and running your first investigation.

Tests

uv run pytest tests/

Tests are fully deterministic — no Splunk connection, no server required. Fixtures live in tests/fixtures/.

Testing the --live path locally

local_splunk/ provides a throwaway single-instance Splunk container (Docker, based on splunk/docker-splunk) for exercising --live queries against a real Splunk REST API and real SPL execution — without production credentials or SSO. See local_splunk/README.md for setup/teardown steps.

Key files

File

Purpose

splunk/config.py

All tunables — thresholds, paths, auth

splunk/parsers.py

parse_splunk_json / parse_splunk_csvpl.DataFrame

splunk/detectors.py

detect_spikes, detect_cert_anomalies, detect_event_pairs/detect_event_pair_patterns (entity-keyed A-precedes-B correlation, e.g. cert error → later handshake failure on the same host), host_error_ranking, detect_slow_queries, detect_numeric_anomalies, etc.

splunk/connector.py

Facade: loading, run state, standalone agent loop, python -m splunk.connector CLI

splunk/mcp_server.py

FastMCP server — 9 investigation tools (thin wrappers over connector.py)

splunk/tui.py

Terminal UI — python -m splunk.tui, reads splunk.db directly

splunk/runner.py

CLI entry point

splunk/client.py

Splunk REST client (cookie-based, SSO-compatible)

splunk/auth.py

Playwright SSO — opens Chromium, saves cookie

splunk/db.py

SQLite store: events, findings, reports, queries, active_runs, alerts, per-sourcetype schema cache

splunk/logger.py

Structured JSON-lines logging per run — audit trail for every connector action

standalone/watcher.py

Standalone python -m standalone.watcher process — polls Splunk on an interval, runs detectors, writes hits to the alerts table (consumed via splunk__check_alerts/splunk__ack_alert)

standalone/agent.py

Standalone LangGraph ReAct agent (--investigate flag) — see Standalone agent

splunk/llm_backends.py

Pluggable chat backend for standalone/agent.pyollama, claude_cli, copilot_cli

splunk/investigation_areas.py

Registry of investigation domains (prompt + SPL template) consumed by standalone/agent.py's tools

Environment variables

Variable

Default

Purpose

SPLUNK_URL

Splunk base URL (required for live queries)

SPLUNK_INDEX

*

Default index substituted into generated follow-up SPL (standalone agent path)

SPLUNK_KNOWN_INDEXES

Comma-separated indexes relevant to your environment — reference only, surfaced to the user during the live-SPL preflight; doesn't affect SPLUNK_INDEX or generated SPL

SPLUNK_INVESTIGATOR_MAX_ITER

3

MCP-driven investigation loop's iteration cap (primary path — splunk__submit_report)

SPLUNK_CORRELATE_WINDOW

60

Event-pair correlation window (seconds) for detect_event_pairs/detect_event_pair_patterns

SPLUNK_SPIKE_THRESHOLD

10

Events/window to trigger a spike

SPLUNK_SPIKE_WINDOW

60

Spike detection window (seconds)

SPLUNK_SLOW_QUERY_THRESHOLD_MS

1000

Duration (ms) above which an event is flagged as a slow query

SPLUNK_ANOMALY_WINDOW

20

Rolling window size (events) for z-score anomaly detection

SPLUNK_ANOMALY_Z_THRESHOLD

3.0

|z-score| above which an event is flagged as a numeric anomaly

SPLUNK_COOKIE_NAME

splunkd_8089

Splunk session cookie name

SPLUNK_AUTH_PATH

~/.splunk/auth.json

Cookie persist path

SPLUNK_POLL_INTERVAL

2

Live REST job polling interval (seconds)

SPLUNK_POLL_TIMEOUT

300

Live REST job poll timeout (seconds)

SPLUNK_MAX_REAUTH

3

Max silent re-auth attempts on a 401 before failing

LOG_LEVEL

DEBUG

Log verbosity

SPLUNK_AGENT_BACKEND

ollama

Standalone agent (--investigate) chat backend — ollama, claude_cli, copilot_cli

SPLUNK_LLM_MODEL

qwen2.5:14b

Ollama model (backend ollama)

SPLUNK_CLAUDE_CLI_MODEL

sonnet

Model passed to claude -p --model (backend claude_cli)

SPLUNK_COPILOT_CLI_MODEL

claude-sonnet-4.5

Model passed to copilot -p --model (backend copilot_cli)

SPLUNK_AGENT_MAX_ITER

10

Standalone agent ReAct loop cap

SPLUNK_WATCH_SPL

SPL query the watcher (standalone/watcher.py) polls on a loop

SPLUNK_WATCH_INTERVAL

60

Seconds between watcher poll cycles

Put these in a .env file at the repo root (gitignored).

Agent instructions

Install Server
F
license - not found
A
quality
B
maintenance

Maintenance

Maintainers
Response time
Release cycle
Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Servers

  • A
    license
    A
    quality
    A
    maintenance
    Enables AI agents to interact seamlessly with Splunk environments through 20+ tools for search, analytics, data discovery, administration, and health monitoring. Features AI-powered troubleshooting workflows and supports multiple Splunk instances with production-ready security.
    53
    27
    Apache 2.0
  • A
    license
    B
    quality
    D
    maintenance
    Enables AI-driven SOC investigations by providing automated Splunk querying, threat intelligence enrichment, and response actions through natural language. Includes tools for IP pivoting, lateral movement detection, and label harvesting.
    31
    1
    Apache 2.0
  • A
    license
    -
    quality
    D
    maintenance
    Enables AI assistants to search, manage, and analyze data in Splunk instances through the Model Context Protocol. Supports SPL queries, index management, alerts, dashboards, and more.
    1
    Apache 2.0

View all related MCP servers

Related MCP Connectors

View all MCP Connectors

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/debaditya-mohankudo/splunk-intelligence'

If you have feedback or need assistance with the MCP directory API, please join our Discord server