Skip to main content
Glama
will-sh

Spark Iceberg MCP Server

by will-sh

Spark Iceberg MCP Server

A Model Context Protocol server that provides read-only access to Apache Iceberg tables via Spark SQL.

Use it from Cursor, Claude Desktop, or any MCP client when you have Iceberg data on HDFS/S3 and want an LLM to inspect schemas and run SELECT queries — without Hive or Impala.

Inspired by cloudera/iceberg-mcp-server (Impala-based).

Tools

Tool

Description

execute_query(query)

Run read-only Spark SQL (SELECT, SHOW, DESCRIBE, WITH, EXPLAIN)

list_namespaces()

SHOW NAMESPACES IN <catalog>

list_tables(namespace)

SHOW TABLES IN <catalog>.<namespace>

describe_table(table)

DESCRIBE TABLE for a table

Results are returned as JSON with columns, rows, and truncated flags.

Related MCP server: Cloudera Iceberg MCP Server

Requirements

  • Python 3.9+

  • Spark 3.x with Iceberg runtime JAR

  • Network/filesystem access to the Iceberg warehouse (e.g. HDFS on a CDP node)

Two execution modes:

Mode

When to use

subprocess (recommended on CDP)

spark3-submit available; set SPARK_SUBMIT_BIN

pyspark

Local dev with uv sync --extra pyspark

Quick start (CDP lab cluster)

Run the MCP server on the cluster node (or any host with spark3-submit + HDFS access):

git clone https://github.com/will-sh/spark-iceberg-mcp-server.git
cd spark-iceberg-mcp-server
uv sync

Set environment variables (example for wxiao-732 lab):

export SPARK_EXECUTION_MODE=subprocess
export SPARK_SUBMIT_BIN=spark3-submit
export SPARK_MASTER="local[2]"
export SPARK_ICEBERG_CATALOG=local
export SPARK_ICEBERG_CATALOG_TYPE=hadoop
export SPARK_ICEBERG_WAREHOUSE="hdfs://ccycloud-1.wxiao-732.root.comops.site:8020/user/systest/iceberg-warehouse"
export SPARK_ICEBERG_RUNTIME_JAR="/opt/cloudera/parcels/CDH-7.3.2-1.cdh7.3.2.p30000.83076434/lib/iceberg/iceberg-spark-runtime-3.5_2.12-*.jar"
export HADOOP_CONF_DIR=/etc/hadoop/conf
export SPARK_MAX_ROWS=1000

Test a query directly:

uv run spark-iceberg-query --query "SELECT count(*) AS n FROM local.datapulse.events"

Start the MCP server:

uv run run-server

Cursor configuration

Add to Cursor MCP settings (~/.cursor/mcp.json or project MCP config):

{
  "mcpServers": {
    "spark-iceberg": {
      "command": "uv",
      "args": [
        "--directory",
        "/path/to/spark-iceberg-mcp-server",
        "run",
        "run-server"
      ],
      "env": {
        "SPARK_EXECUTION_MODE": "subprocess",
        "SPARK_SUBMIT_BIN": "spark3-submit",
        "SPARK_MASTER": "local[2]",
        "SPARK_ICEBERG_CATALOG": "local",
        "SPARK_ICEBERG_CATALOG_TYPE": "hadoop",
        "SPARK_ICEBERG_WAREHOUSE": "hdfs://ccycloud-1.wxiao-732.root.comops.site:8020/user/systest/iceberg-warehouse",
        "SPARK_ICEBERG_RUNTIME_JAR": "/opt/cloudera/parcels/CDH-7.3.2-1.cdh7.3.2.p30000.83076434/lib/iceberg/iceberg-spark-runtime-3.5_2.12-*.jar",
        "HADOOP_CONF_DIR": "/etc/hadoop/conf",
        "SPARK_MAX_ROWS": "100"
      }
    }
  }
}

Or install from GitHub with uvx:

{
  "mcpServers": {
    "spark-iceberg": {
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/will-sh/spark-iceberg-mcp-server@main",
        "run-server"
      ],
      "env": {
        "SPARK_EXECUTION_MODE": "subprocess",
        "SPARK_SUBMIT_BIN": "spark3-submit",
        "SPARK_ICEBERG_WAREHOUSE": "hdfs://your-nn:8020/user/systest/iceberg-warehouse",
        "SPARK_ICEBERG_RUNTIME_JAR": "/opt/cloudera/parcels/CDH/lib/iceberg/iceberg-spark-runtime-3.5_2.12-*.jar"
      }
    }
  }
}

Note: uvx still needs spark3-submit and HDFS access on the host where Cursor launches the MCP process. For remote CDP clusters, run Cursor's MCP on the cluster node via SSH Remote, or use a bastion with Hadoop client configs.

Environment variables

Variable

Default

Description

SPARK_EXECUTION_MODE

auto

subprocess, pyspark, or auto (subprocess if SPARK_SUBMIT_BIN set)

SPARK_SUBMIT_BIN

spark3-submit

Spark submit binary for subprocess mode

SPARK_MASTER

local[2]

Spark master URL (local[2], yarn, etc.)

SPARK_ICEBERG_CATALOG

local

Iceberg catalog name

SPARK_ICEBERG_CATALOG_TYPE

hadoop

Catalog type (hadoop, hive, rest, …)

SPARK_ICEBERG_WAREHOUSE

Warehouse path (required for Hadoop catalog)

SPARK_ICEBERG_RUNTIME_JAR

Path to iceberg-spark-runtime JAR (glob supported)

SPARK_HOME

Optional SPARK_HOME for subprocess

HADOOP_CONF_DIR

Hadoop config directory

SPARK_MAX_ROWS

1000

Max rows returned per query

MCP_TRANSPORT

stdio

MCP transport: stdio, http, or sse

Security

  • Only read-only SQL prefixes are allowed (SELECT, SHOW, DESCRIBE, WITH, EXPLAIN).

  • This is a basic guardrail; do not expose the MCP server to untrusted networks without additional controls.

  • Prefer a dedicated read-only HDFS/Spark principal in production.

Development

uv sync --group dev
uv run pytest
uv run ruff check src tests

License

Apache License 2.0 — see LICENSE.

Related MCP Connectors

Related MCP Servers

  • A
    license
    B
    quality
    D
    maintenance
    Enables read-only access to Apache Iceberg tables via Impala, allowing LLMs to inspect database schemas and execute SQL queries to retrieve data from Iceberg tables.
    2
    14
    Apache 2.0
  • A
    license
    C
    quality
    D
    maintenance
    Enables natural language interaction with Apache Iceberg Lakehouse tables through MCP, supporting read-only operations like listing namespaces, tables, schemas, and partitions.
    5
    Apache 2.0
  • A
    license
    A
    quality
    C
    maintenance
    Provides read-only SQL access to Apache Iceberg tables via HiveServer2, enabling querying, schema discovery, and database listing on Cloudera Data Platform.
    3
    Apache 2.0