Databricks MCP Server

# Databricks MCP Server A FastAPI-based MCP (Model Context Protocol) server that provides tools for local file management and Databricks operations. ## Features ### Local File Management - Create folders and directories - Create Python files with content - Edit existing files ### Databricks Operations - Submit code to Databricks clusters - Create and run Databricks jobs - Create Delta Live Tables (DLT) pipelines - Get job errors and status information ## Setup ### 1. Install Dependencies ```bash pip install -r requirements.txt ``` ### 2. Environment Configuration Create a `.env` file in the project root: ```env DATABRICKS_HOST=https://e2-demo-west.cloud.databricks.com/ DATABRICKS_TOKEN=YOUR_DAPI_TOKEN ``` ### 3. Run the Server ```bash uvicorn main:app --reload --host 0.0.0.0 --port 8000 ``` ## API Endpoints ### File Management #### Create Folder ```bash POST /file/create_folder { "path": "/path/to/folder" } ``` #### Create Python File ```bash POST /file/create_py_file { "path": "/path/to/file.py", "content": "print('Hello, World!')" } ``` #### Edit File ```bash POST /file/edit_file { "path": "/path/to/file.py", "content": "print('Updated content')" } ``` ### Databricks Operations #### Submit Code ```bash POST /databricks/submit_code { "code": "print('Hello from Databricks!')", "cluster_id": "your-cluster-id" } ``` #### Create Job ```bash POST /databricks/create_job { "job_config": { "name": "My Job", "new_cluster": { "spark_version": "11.3.x-scala2.12", "node_type_id": "i3.xlarge", "num_workers": 1 }, "notebook_task": { "notebook_path": "/Users/your.email@databricks.com/your_notebook" } } } ``` #### Run Job ```bash POST /databricks/run_job { "job_id": "your-job-id" } ``` #### Create DLT Pipeline ```bash POST /databricks/create_dlt_pipeline { "pipeline_config": { "name": "My DLT Pipeline", "storage": "dbfs:/pipelines/storage", "clusters": [{"label": "default", "num_workers": 1}], "libraries": [{"notebook": {"path": "/Users/your.email@databricks.com/your_dlt_notebook"}}] } } ``` #### Get Job Error ```bash POST /databricks/get_job_error { "run_id": "your-run-id" } ``` #### Check Job Status ```bash POST /databricks/check_job_status { "job_id": "your-job-id", "run_id": "your-run-id" } ``` ## Claude Desktop Integration ### 1. Copy Configuration Files Copy the MCP configuration files to your Claude Desktop configuration directory: **macOS:** ```bash cp mcp_server_config.json ~/Library/Application\ Support/Claude/claude_desktop_config.json cp mcp_tools.json ~/Library/Application\ Support/Claude/mcp_tools.json ``` **Windows:** ```bash copy mcp_server_config.json %APPDATA%\Claude\claude_desktop_config.json copy mcp_tools.json %APPDATA%\Claude\mcp_tools.json ``` **Linux:** ```bash cp mcp_server_config.json ~/.config/Claude/claude_desktop_config.json cp mcp_tools.json ~/.config/Claude/mcp_tools.json ``` ### 2. Start Your MCP Server Make sure your Databricks MCP server is running: ```bash uvicorn main:app --host 0.0.0.0 --port 8000 ``` ### 3. Restart Claude Desktop Restart Claude Desktop to load the new MCP configuration. ### 4. Use the Tools Claude Desktop will now have access to all the Databricks and file management tools. You can ask Claude to: - "Create a folder called 'my_project'" - "Create a Python file with some Databricks code" - "Submit this code to my Databricks cluster" - "Create a DLT pipeline for data processing" - "Check the status of my job" ## Testing Run the test suite: ```bash pytest test_main.py -v ``` ## Error Handling The server provides detailed error messages and logging. All operations return a consistent response format: ```json { "status": "success|error", "message": "Description of what happened", "detail": "Error details (if status is error)" } ``` ## Security Notes - Store your Databricks token securely - Use environment variables for sensitive configuration - Consider using Databricks workspace-specific tokens with limited permissions - The server runs on `0.0.0.0:8000` by default - adjust for your security requirements ## Troubleshooting ### Common Issues 1. **Environment Variables Not Loaded**: Make sure you have `python-dotenv` installed and a `.env` file in the project root. 2. **Databricks Connection Issues**: Verify your host URL and token are correct. Test with a simple API call first. 3. **Permission Errors**: Ensure the server has write permissions for file operations. 4. **Port Already in Use**: Change the port in the uvicorn command or kill the existing process. ### Logs The server provides detailed logging. Check the console output for debugging information. --- ## 1. Create the Subfolder and Minimal Server **Directory structure:** ``` mcp_test_server/ ├── main.py └── requirements.txt ``` **main.py:** ```python from fastapi import FastAPI app = FastAPI() @app.get("/") def root(): return {"status": "ok", "message": "MCP test server is running"} ``` **requirements.txt:** ``` fastapi uvicorn ``` --- ## 2. Install Dependencies ```sh cd mcp_test_server python3 -m venv venv source venv/bin/activate pip install -r requirements.txt ``` --- ## 3. Test the Server Manually ```sh venv/bin/python -m uvicorn main:app --host 0.0.0.0 --port 9000 ``` - Visit [http://localhost:9000/](http://localhost:9000/) in your browser. You should see: ```json {"status": "ok", "message": "MCP test server is running"} ``` --- ## 4. Update Claude Config Edit your `claude_desktop_config.json` to point to this test server: ```json { "mcpServers": { "test-mcp": { "command": "/full/path/to/mcp_test_server/venv/bin/python", "args": ["-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "9000"] } } } ``` - Replace `/full/path/to/` with the actual path on your system. --- ## 5. Restart Claude Desktop - Fully quit and restart Claude Desktop. - See if you get any connection or server errors. --- ## 6. What to Look For - If Claude Desktop can connect, you should see requests in your test server’s terminal. - If not, and you still see no requests, the issue is with Claude’s config, file location, or local network. --- ## 7. If You Want, I Can Generate the Files Let me know if you want me to generate the exact code for `main.py` and `requirements.txt` for you! --- **This minimal test will help you isolate whether the problem is with your main server, your config, or Claude Desktop’s ability to reach any local MCP server.** Let me know when you’ve tried this and what you see! --- ## Why This Happens - **Claude Desktop starts the server in the directory:** `/Users/stephen.hsu/Desktop/cursor_test/mcp_dbx_pure` - **But your test server code is in:** `/Users/stephen.hsu/Desktop/cursor_test/mcp_dbx_pure/mcp_test_server/main.py` So, when it runs: ```sh venv/bin/python -m uvicorn main:app --host 0.0.0.0 --port 9000 ``` from the parent directory, there is **no `main.py` in the current directory**, so import fails. --- ## How to Fix ### 1. Update the `"args"` in your config to include the subfolder: Change from: ```json "args": ["-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "9000"] ``` to: ```json { "mcpServers": { "test-mcp": { "command": "/Users/stephen.hsu/Desktop/cursor_test/mcp_dbx_pure/mcp_test_server/venv/bin/python", "args": ["-m", "uvicorn", "mcp_test_server.main:app", "--host", "0.0.0.0", "--port", "9000"] } } } ``` This tells Uvicorn to look for the `main` module inside the `mcp_test_server` package. --- ### 2. Make `mcp_test_server` a Python package Add an empty `__init__.py` file to `mcp_test_server`: ```sh touch mcp_test_server/__init__.py ``` --- ### 3. Update Your Config Your `claude_desktop_config.json` should look like: ```json { "mcpServers": { "test-mcp": { "command": "/Users/stephen.hsu/Desktop/cursor_test/mcp_dbx_pure/mcp_test_server/venv/bin/python", "args": ["-m", "uvicorn", "mcp_test_server.main:app", "--host", "0.0.0.0", "--port", "9000"] } } } ``` --- ### 4. Restart Claude Desktop - Save the config. - Restart Claude Desktop. --- ## Why This Works - `uvicorn mcp_test_server.main:app` tells Python to look for `main.py` inside the `mcp_test_server` package, no matter what the current working directory is. - The `__init__.py` file makes `mcp_test_server` a valid Python package. --- ## Summary - Update the `"args"` to use `mcp_test_server.main:app` - Add `__init__.py` to `mcp_test_server` - Restart Claude Desktop This should allow Claude Desktop to start and connect to your minimal MCP test server! Let me know if you see requests in your test server terminal after this change, or if you get a new error.