awesome-databricks-mcp
Host Model Context Protocol (MCP) prompts and tools on Databricks Apps, enabling AI assistants like Claude to interact with your Databricks workspace through a secure, authenticated interface.
What is this?
This template lets you create an MCP server that runs on Databricks Apps. You can:
- 📝 Add prompts as simple markdown files in the
prompts/ folder - 🛠️ Create tools as Python functions that leverage Databricks SDK
- 🔐 Authenticate securely with OAuth through Databricks Apps
- 🚀 Deploy instantly to make your MCP server accessible to Claude
- 🖥️ Web Interface with a modern React TypeScript frontend for MCP discovery
- 🧪 Comprehensive Testing with automated MCP validation tools
Think of it as a bridge between Claude and your Databricks workspace - you define what Claude can see and do, and this server handles the rest.
How it Works
Architecture Overview
┌─────────────┐ MCP Protocol ┌──────────────────┐ OAuth ┌─────────────────┐
│ Claude │ ◄─────────────────────► │ dba-mcp-proxy │ ◄──────────────────► │ Databricks App │
│ CLI │ (stdio/JSON-RPC) │ (local process) │ (HTTPS/SSE) │ (MCP Server) │
└─────────────┘ └──────────────────┘ └─────────────────┘
▲ │
│ ▼
└────────── Databricks OAuth ──────► Workspace APIs
Components
- MCP Server (
server/app.py): A FastAPI app with integrated MCP server that:- Dynamically loads prompts from
prompts/*.md files - Exposes Python functions as MCP tools via modular tool system
- Handles both HTTP requests and MCP protocol over Server-Sent Events
- Uses FastMCP framework for seamless MCP integration
- React Frontend (
client/): A modern TypeScript React application that:- Provides a web interface for MCP discovery and testing
- Shows available prompts, tools, and MCP configuration
- Includes copy-paste setup commands for Claude integration
- Built with TailwindCSS, Radix UI, and modern React patterns
- Uses Vite for fast development and building
- Prompts (
prompts/): Simple markdown files where:- Filename = prompt name (e.g.,
build_dlt_pipeline.md → build_dlt_pipeline prompt) - First line with
# = description - File content = what gets returned to Claude
- Modular Tools System (
server/tools/): Organized tool modules that:- Break down functionality into logical, manageable components
- Provide 104+ tools across 9 specialized modules
- Enable better maintainability and collaboration
- Support easy addition of new tools
- Local Proxy (
dba_mcp_proxy/): Authenticates and proxies MCP requests:- Handles Databricks OAuth authentication automatically
- Translates between Claude's stdio protocol and HTTP/SSE
- Works with both local development and deployed apps
🎬 Demo
This 5-minute video shows you how to set up and use the MCP server with Claude and/or Cursor: https://www.youtube.com/watch?v=_yPtm2iH04o
Quick Start
Create Your Own MCP Server
Step 1: Install Databricks CLI and Set Up GitHub SSH Access
Before you begin, make sure you have the Databricks CLI and GitHub SSH access configured.
Install Databricks CLI:
macOS:
# Using Homebrew (recommended)
brew install databricks/tap/databricks
# Using pip
pip install databricks-cli
# Verify installation
databricks --version
Windows:
# Using pip
pip install databricks-cli
# Or download from official releases
# https://github.com/databricks/databricks-cli/releases
# Download the .exe file and add to PATH
# Verify installation
databricks --version
Linux:
# Using pip
pip install databricks-cli
# Using apt (Ubuntu/Debian)
curl -fsSL https://databricks.com/install-cli.sh | bash
# Using yum (RHEL/CentOS)
curl -fsSL https://databricks.com/install-cli.sh | bash
# Verify installation
databricks --version
Set Up GitHub SSH Access:
macOS:
# Generate SSH key (if you don't have one)
ssh-keygen -t ed25519 -C "your-email@example.com"
# Start ssh-agent
eval "$(ssh-agent -s)"
# Add SSH key to ssh-agent
ssh-add ~/.ssh/id_ed25519
# Copy public key to clipboard
pbcopy < ~/.ssh/id_ed25519.pub
# Add to GitHub: https://github.com/settings/keys
# Click "New SSH key" and paste the copied key
Windows:
# Generate SSH key (if you don't have one)
ssh-keygen -t ed25519 -C "your-email@example.com"
# Start ssh-agent (PowerShell as Administrator)
Set-Service ssh-agent -StartupType Automatic
Start-Service ssh-agent
# Add SSH key to ssh-agent
ssh-add ~/.ssh/id_ed25519
# Copy public key to clipboard
Get-Content ~/.ssh/id_ed25519.pub | Set-Clipboard
# Add to GitHub: https://github.com/settings/keys
# Click "New SSH key" and paste the copied key
Linux:
# Generate SSH key (if you don't have one)
ssh-keygen -t ed25519 -C "your-email@example.com"
# Start ssh-agent
eval "$(ssh-agent -s)"
# Add SSH key to ssh-agent
ssh-add ~/.ssh/id_ed25519
# Copy public key to clipboard (if xclip is available)
xclip -sel clip < ~/.ssh/id_ed25519.pub
# Or display the key to copy manually
cat ~/.ssh/id_ed25519.pub
# Add to GitHub: https://github.com/settings/keys
# Click "New SSH key" and paste the copied key
Test GitHub SSH Connection:
# Test the connection
ssh -T git@github.com
# You should see: "Hi username! You've successfully authenticated..."
Configure Git with SSH:
# Set your Git username and email
git config --global user.name "Your Name"
git config --global user.email "your-email@example.com"
# Verify SSH is working by cloning a private repo
git clone git@github.com:username/private-repo.git
Step 2: Clone the Repo
# Clone your new repository
git clone https://github.com/PulkitXChadha/awesome-databricks-mcp.git
cd awesome-databricks-mcp
# Run the interactive setup
./setup.sh
This will:
- Configure Databricks authentication
- Set your MCP server name
- Install all dependencies (Python + Node.js)
- Create your
.env.local file
Step 3: Deploy with Claude
In Claude Code, run:
This will:
- Deploy your MCP server to Databricks Apps
- Configure the MCP integration
- Show you available prompts and tools
Then restart Claude Code to use your new MCP server.
Add to Claude CLI
After deployment, add your MCP server to Claude:
# Set your Databricks configuration
export DATABRICKS_HOST="https://your-workspace.cloud.databricks.com"
export DATABRICKS_APP_URL="https://your-app.databricksapps.com" # Get this from ./app_status.sh
export SERVER_NAME="your-server-name" # This comes from config.yaml (set during ./setup.sh)
# Add your MCP server to Claude (user-scoped)
claude mcp add $SERVER_NAME --scope user -- \
uvx --from git+ssh://git@github.com/YOUR-USERNAME/your-repo.git dba-mcp-proxy \
--databricks-host $DATABRICKS_HOST \
--databricks-app-url $DATABRICKS_APP_URL
Local Development
# Clone and setup
git clone <your-repo>
cd <your-repo>
./setup.sh
# Start dev server (both backend and frontend)
./watch.sh
# Set your configuration for local testing
export DATABRICKS_HOST="https://your-workspace.cloud.databricks.com"
export DATABRICKS_APP_URL="http://localhost:8000" # Local dev server
# Add to Claude for local testing
claude mcp add databricks-mcp-local --scope local -- \
uvx --from git+ssh://git@github.com/YOUR-ORG/YOUR-REPO.git dba-mcp-proxy \
--databricks-host $DATABRICKS_HOST \
--databricks-app-url $DATABRICKS_APP_URL
Running Locally
Prerequisites
Before running the MCP server locally, ensure you have:
- Python 3.11+ and Node.js 18+ installed
- Databricks CLI configured with
databricks auth login - Git for cloning the repository
- uv package manager (recommended) or pip for Python dependencies
- bun (recommended) or npm for Node.js dependencies
Step-by-Step Local Setup
# Clone your repository
git clone https://github.com/PulkitXChadha/awesome-databricks-mcp.git
cd awesome-databricks-mcp
# Run the interactive setup script
./setup.sh
The setup script will:
- Install Python dependencies using
uv or pip - Install Node.js dependencies using
bun or npm - Configure your Databricks workspace settings
- Create a
.env.local file with your configuration
2. Start the Development Server
# Start both backend (FastAPI) and frontend (React) servers
./watch.sh
This command starts:
- Backend: FastAPI server on
http://localhost:8000 - Frontend: React development server on
http://localhost:3000 - File watching: Automatic reloading when files change
3. Verify Local Setup
Open your browser and navigate to:
- Backend API: http://localhost:8000/docs (FastAPI Swagger UI)
- Frontend: http://localhost:3000 (React application)
- MCP Endpoint: http://localhost:8000/mcp/ (MCP server)
4. Test with Claude CLI
# Set environment variables for local testing
export DATABRICKS_HOST="https://your-workspace.cloud.databricks.com"
export DATABRICKS_APP_URL="http://localhost:8000"
# Add the local MCP server to Claude
claude mcp add databricks-mcp-local --scope local -- \
uvx --from git+ssh://git@github.com/PulkitXChadha/awesome-databricks-mcp.git dba-mcp-proxy \
--databricks-host $DATABRICKS_HOST \
--databricks-app-url $DATABRICKS_APP_URL
# Test the connection
echo "What MCP prompts are available from databricks-mcp-local?" | claude
Development Workflow
Making Changes
- Edit prompts: Modify files in
prompts/ directory - Edit tools: Update functions in appropriate modules under
server/tools/ - Edit frontend: Modify React components in
client/src/ - Edit backend: Update FastAPI routes in
server/
All changes automatically reload thanks to the file watchers in ./watch.sh.
Testing Changes
# Test local MCP server directly
./claude_scripts/test_local_mcp_curl.sh
# Test with MCP proxy
./claude_scripts/test_local_mcp_proxy.sh
# Use the web-based MCP Inspector
./claude_scripts/inspect_local_mcp.sh
Debugging
- Backend logs: Check terminal output from
./watch.sh - Frontend logs: Check browser console and terminal output
- MCP logs: Monitor the
/mcp/ endpoint responses - Database queries: Check Databricks workspace logs
Local vs Production Differences
| Feature | Local Development | Production |
|---|
| Authentication | Databricks CLI token | OAuth via Databricks Apps |
| URL | http://localhost:8000 | https://your-app.databricksapps.com |
| HTTPS | No (HTTP only) | Yes (HTTPS required) |
| File watching | Yes (auto-reload) | No |
| Debug mode | Yes | No |
| Logs | Terminal output | Databricks Apps logs |
Troubleshooting Local Issues
Common Problems
Port conflicts:
# Check what's using port 8000
lsof -i :8000
# Kill process if needed
kill -9 <PID>
Dependencies not found:
# Reinstall Python dependencies
uv sync
# Reinstall Node.js dependencies
cd client && bun install
Databricks authentication:
# Refresh Databricks CLI credentials
databricks auth login
# Verify configuration
databricks config get
MCP connection issues:
# Test MCP endpoint directly
curl http://localhost:8000/mcp/
# Check Claude MCP configuration
claude mcp list
- Use
uv instead of pip for faster Python dependency management - Use
bun instead of npm for faster Node.js dependency management - The
./watch.sh script uses uvicorn --reload for fast backend development - Frontend uses Vite for fast hot module replacement
Deployment
# Deploy to Databricks Apps
./deploy.sh
# Check status and get your app URL
./app_status.sh
Your MCP server will be available at https://your-app.databricksapps.com/mcp/
The app_status.sh script will show your deployed app URL, which you'll need for the DATABRICKS_APP_URL environment variable when adding the MCP server to Claude.
Authentication
- Local Development: No authentication required
- Production: OAuth is handled automatically by the proxy using your Databricks CLI credentials
Examples
Using with Claude
Once added, you can interact with your MCP server in Claude:
Human: What prompts are available?
Claude: I can see the following prompts from your Databricks MCP server:
- build_ldp_pipeline: Build a DLT pipeline for data processing
Human: Can you execute a SQL query to show databases?
Claude: I'll execute that SQL query for you using the execute_dbsql tool.
[Executes SQL and returns results]
Project Structure
├── server/ # FastAPI backend with MCP server
│ ├── app.py # Main application + MCP server setup
│ ├── tools/ # Modular MCP tools implementation
│ │ ├── __init__.py # Tool registration and loading
│ │ ├── core.py # Core and health tools
│ │ ├── sql_operations.py # SQL and warehouse tools
│ │ ├── unity_catalog.py # Unity Catalog tools
│ │ ├── data_management.py # Data and DBFS tools
│ │ ├── jobs_pipelines.py # Jobs and DLT pipeline tools
│ │ ├── workspace_files.py # Workspace file tools
│ │ ├── dashboards.py # Dashboard tools
│ │ ├── repositories.py # Git repository tools
│ │ └── governance.py # Governance and compliance tools
│ └── routers/ # API endpoints
├── client/ # React TypeScript frontend
│ ├── src/ # Source code
│ │ ├── components/ # Reusable UI components
│ │ ├── pages/ # Page components
│ │ └── fastapi_client/ # Auto-generated API client
│ ├── package.json # Node.js dependencies
│ └── tailwind.config.js # TailwindCSS configuration
├── prompts/ # MCP prompts (markdown files)
│ └── build_dlt_pipeline.md # DLT pipeline building prompt
├── dba_mcp_proxy/ # MCP proxy for Claude CLI
│ └── mcp_client.py # OAuth + proxy implementation
├── claude_scripts/ # Comprehensive testing tools
│ ├── test_local_mcp_*.sh # Local MCP testing scripts
│ ├── test_remote_mcp_*.sh # Remote MCP testing scripts
│ ├── test_uc_tools.py # Unity Catalog tools testing
│ └── inspect_*.sh # Web-based MCP Inspector
├── docs/ # Documentation
│ ├── databricks_apis/ # Databricks API documentation
│ └── unity_catalog_tools.md # Unity Catalog tools documentation
├── scripts/ # Development tools
└── pyproject.toml # Python package configuration
Advanced Usage
Environment Variables
Configure in .env.local:
DATABRICKS_HOST=https://your-workspace.cloud.databricks.com
DATABRICKS_TOKEN=your-token # For local development
DATABRICKS_SQL_WAREHOUSE_ID=your-warehouse-id # For SQL tools
Tools can access the full Databricks SDK:
@mcp_server.tool
def create_job(name: str, notebook_path: str, cluster_id: str) -> dict:
"""Create a Databricks job."""
w = get_workspace_client()
job = w.jobs.create(
name=name,
tasks=[{
"task_key": "main",
"notebook_task": {"notebook_path": notebook_path},
"existing_cluster_id": cluster_id
}]
)
return {"job_id": job.job_id, "run_now_url": f"{DATABRICKS_HOST}/#job/{job.job_id}"}
Testing Your MCP Server
This template includes comprehensive testing tools for validating MCP functionality at multiple levels.
Quick Verification
After adding the MCP server to Claude, verify it's working:
# List available prompts and tools
echo "What MCP prompts are available from databricks-mcp?" | claude
# Test a specific prompt
echo "Use the build_dlt_pipeline prompt from databricks-mcp" | claude
Comprehensive Testing Suite
The claude_scripts/ directory contains 6 testing tools for thorough MCP validation:
Command Line Tests
# Test local MCP server (requires ./watch.sh to be running)
./claude_scripts/test_local_mcp_curl.sh # Direct HTTP/curl tests with session handling
./claude_scripts/test_local_mcp_proxy.sh # MCP proxy client tests
# Test remote MCP server (requires Databricks auth and deployment)
./claude_scripts/test_remote_mcp_curl.sh # OAuth + HTTP tests with dynamic URL discovery
./claude_scripts/test_remote_mcp_proxy.sh # Full end-to-end MCP proxy tests
Interactive Web UI Tests
# Launch MCP Inspector for visual testing (requires ./watch.sh for local)
./claude_scripts/inspect_local_mcp.sh # Local server web interface
./claude_scripts/inspect_remote_mcp.sh # Remote server web interface
MCP Inspector Features:
- 🖥️ Web-based interface for interactive MCP server testing
- 🔧 Visual tool execution with parameter input forms
- 📊 Real-time request/response monitoring
- 🐛 Protocol-level debugging and error inspection
- 📋 Complete tool and resource discovery
What Each Test Validates
| Test Type | Authentication | Protocol | Session Management | Tool Discovery |
|---|
| curl tests | ✅ | ✅ | ✅ | ✅ |
| proxy tests | ✅ | ✅ | ✅ | ✅ |
| MCP Inspector | ✅ | ✅ | ✅ | ✅ |
All tests dynamically discover app URLs and handle OAuth authentication automatically.
See claude_scripts/README.md for detailed documentation.
Web Interface Testing
The React frontend provides an additional way to test your MCP server:
# Start the development server
./watch.sh
# Open http://localhost:3000 in your browser
# Navigate to the MCP Discovery page to see:
# - Available prompts and tools
# - MCP configuration details
# - Copy-paste setup commands for Claude
Troubleshooting
- Authentication errors: Run
databricks auth login to refresh credentials - MCP not found: Ensure the app is deployed and accessible
- Tool errors: Check logs at
https://your-app.databricksapps.com/logz - MCP connection issues:
- Check Claude logs:
tail -f ~/Library/Logs/Claude/*.log - Verify the proxy works:
uvx --from git+ssh://... dba-mcp-proxy --help - Test with echo pipe:
echo "list your mcp commands" | claude
- Cached version issues: If you get errors about missing arguments after an update:
# Clear uvx cache for this package
rm -rf ~/.cache/uv/git-v0/checkouts/*/
# Or clear entire uv cache
uv cache clean
- Frontend build issues: Ensure Node.js dependencies are installed:
Contributing
- Fork the repository
- Add your prompts and tools
- Test locally with
./watch.sh - Submit a pull request
License
See LICENSE.md