# Hue MCP Server
An MCP (Model Context Protocol) server that exposes [HueClientRest](https://github.com/SpanishST/HueClientRest) functionality, allowing AI assistants to interact with Hadoop Hue for executing SQL queries and managing HDFS files.
## What is This?
This server enables AI assistants (like GitHub Copilot, Claude Desktop, or other MCP-compatible clients) to:
- Execute SQL queries on Hadoop Hue using Hive, SparkSQL, or Impala
- Manage HDFS files (list, upload, download)
- Export query results to CSV files
- Browse and manage directory structures
The Model Context Protocol (MCP) is an open standard for connecting AI assistants to external tools and data sources, making them more powerful and context-aware.
## Features
- **SQL Query Execution**: Execute queries using Hive, SparkSQL, or Impala dialects
- **Result Export**: Save query results to CSV files with automatic retry on large datasets
- **HDFS Operations**: List, upload, and download files from HDFS
- **Directory Management**: Check directory existence and browse file structures
- **Robust Error Handling**: Built-in retry mechanisms and detailed error reporting
## Prerequisites
Before installing this MCP server, you need:
1. **Python 3.10 or higher** - [Download Python](https://www.python.org/downloads/)
2. **Astral uv** - Fast Python package installer and environment manager
3. **Visual Studio Code** - For MCP integration with GitHub Copilot
4. **GitHub Copilot subscription** - Required for VS Code MCP integration
5. **Access to a Hadoop Hue server** - You'll need the host URL, username, and password
## Dependencies
This project uses the following key dependencies:
- **[Astral uv](https://github.com/astral-sh/uv)** - An extremely fast Python package and project manager, written in Rust. It's 10-100x faster than pip and handles dependency resolution much better.
- **[mcp[cli]](https://github.com/modelcontextprotocol/python-sdk)** - The official Python SDK for the Model Context Protocol, including CLI tools
- **[hueclientrest](https://github.com/SpanishST/HueClientRest)** - Python client library for interacting with Hadoop Hue REST API
- **[pydantic](https://docs.pydantic.dev/)** - Data validation using Python type annotations
## Installation
### Step 1: Install Astral uv
uv is a modern, fast Python package manager that we use for dependency management.
**On Windows (PowerShell):**
```powershell
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
```
**On macOS/Linux:**
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```
After installation, restart your terminal or add uv to your PATH as instructed by the installer.
Verify installation:
```bash
uv --version
```
### Step 2: Clone and Install the Project
```bash
# Clone the repository
git clone <your-repo-url>
cd hueclientrest-mpc
# Install dependencies and create virtual environment
uv sync
```
The `uv sync` command will:
- Create a virtual environment (`.venv`)
- Install all dependencies from `pyproject.toml`
- Set up the project for development
### Alternative: Using pip
If you prefer pip over uv:
```bash
pip install -e .
```
However, uv is strongly recommended for better performance and dependency management.
## Configuration
### Environment Variables
The server requires the following environment variables to connect to your Hue server:
| Variable | Required | Description |
|----------|----------|-------------|
| `HUE_HOST` | Yes | Hue server URL (e.g., `https://hue.example.com`) |
| `HUE_USERNAME` | Yes | Username for Hue authentication |
| `HUE_PASSWORD` | Yes | Password for Hue authentication |
| `HUE_VERIFY_SSL` | No | Verify SSL certificates (default: `true`) |
| `HUE_SSL_WARNINGS` | No | Show SSL warnings (default: `false`) |
### Setting Up Environment Variables
**Option 1: Using .env file (Recommended for local development)**
```bash
# Create a .env file in the project root
HUE_HOST=https://your-hue-server.com
HUE_USERNAME=your_username
HUE_PASSWORD=your_password
HUE_VERIFY_SSL=true
HUE_SSL_WARNINGS=false
```
**Option 2: System environment variables**
On Windows (PowerShell):
```powershell
$env:HUE_HOST="https://your-hue-server.com"
$env:HUE_USERNAME="your_username"
$env:HUE_PASSWORD="your_password"
```
On macOS/Linux:
```bash
export HUE_HOST="https://your-hue-server.com"
export HUE_USERNAME="your_username"
export HUE_PASSWORD="your_password"
```
## VS Code Integration with GitHub Copilot
### Prerequisites for VS Code Integration
1. **Visual Studio Code** - [Download VS Code](https://code.visualstudio.com/)
2. **GitHub Copilot extension** - Install from VS Code marketplace
3. **GitHub Copilot subscription** - Required for MCP support
4. **This MCP server installed and configured**
### Step 1: Locate Your MCP Configuration File
The MCP configuration file location depends on your operating system:
- **Windows**: `%APPDATA%\Code\User\mcp.json`
- Full path: `C:\Users\<YourUsername>\AppData\Roaming\Code\User\mcp.json`
- **macOS**: `~/Library/Application Support/Code/User/mcp.json`
- **Linux**: `~/.config/Code/User/mcp.json`
If the file doesn't exist, create it.
### Step 2: Configure the MCP Server in VS Code
Add the following configuration to your `mcp.json` file:
```json
{
"mcpServers": {
"hue": {
"command": "uv",
"args": [
"run",
"--directory",
"C:\\Projects\\hueclientrest-mpc",
"hue-mcp-server"
],
"env": {
"HUE_HOST": "https://your-hue-server.com",
"HUE_USERNAME": "your_username",
"HUE_PASSWORD": "your_password",
"HUE_VERIFY_SSL": "true",
"HUE_SSL_WARNINGS": "false"
}
}
}
}
```
**Important Notes:**
- Replace `C:\\Projects\\hueclientrest-mpc` with the actual path to your project
- On Windows, use double backslashes (`\\`) or forward slashes (`/`) in paths
- Replace the environment variable values with your actual Hue credentials
- The `command` is `uv` which will use the uv package manager to run the server
### Step 3: Verify the Configuration
1. **Restart VS Code** completely (close all windows)
2. **Open GitHub Copilot Chat** (Ctrl+Shift+I or Cmd+Shift+I)
3. **Check for the Hue MCP tools**: Type `@workspace` and look for Hue-related capabilities
4. **Test the connection**: Ask Copilot to "list files in HDFS directory /user"
### Step 4: Using the MCP Server with Copilot
Once configured, you can ask GitHub Copilot to interact with your Hue server:
**Example queries:**
- "Execute a Hive query to show tables"
- "List files in HDFS directory /user/data"
- "Download the file /user/data/results.csv from HDFS"
- "Execute this SQL query and save results to CSV: SELECT * FROM my_table LIMIT 100"
### Troubleshooting VS Code Integration
**Issue: MCP server not appearing in Copilot**
- Verify the `mcp.json` path is correct
- Check that uv is installed and in your PATH
- Restart VS Code completely
- Check VS Code's Output panel (View > Output) and select "GitHub Copilot" from the dropdown
**Issue: Authentication errors**
- Verify your HUE_HOST, HUE_USERNAME, and HUE_PASSWORD are correct
- Test connectivity to your Hue server directly
- Check if SSL verification is causing issues (try setting HUE_VERIFY_SSL to false for testing)
**Issue: Command not found**
- Ensure uv is installed: run `uv --version` in terminal
- Verify the project path in mcp.json is correct and uses proper escaping
- Make sure you ran `uv sync` in the project directory
### Alternative: Using Absolute Python Path
If uv is not working or not in your PATH, you can use the absolute path to the Python interpreter:
**Windows example:**
```json
{
"mcpServers": {
"hue": {
"command": "C:\\Projects\\hueclientrest-mpc\\.venv\\Scripts\\python.exe",
"args": ["-m", "hue_mcp_server.server"],
"env": {
"HUE_HOST": "https://your-hue-server.com",
"HUE_USERNAME": "your_username",
"HUE_PASSWORD": "your_password"
}
}
}
}
```
**macOS/Linux example:**
```json
{
"mcpServers": {
"hue": {
"command": "/full/path/to/hueclientrest-mpc/.venv/bin/python",
"args": ["-m", "hue_mcp_server.server"],
"env": {
"HUE_HOST": "https://your-hue-server.com",
"HUE_USERNAME": "your_username",
"HUE_PASSWORD": "your_password"
}
}
}
}
```
## Other Usage Methods
### Claude Desktop Integration
If you're using Claude Desktop instead of VS Code, add this to your Claude config (`~/.config/claude/claude_desktop_config.json` on Mac/Linux or `%APPDATA%\Claude\claude_desktop_config.json` on Windows):
```json
{
"mcpServers": {
"hue": {
"command": "uv",
"args": ["run", "--directory", "/path/to/hueclientrest-mpc", "hue-mcp-server"],
"env": {
"HUE_HOST": "https://your-hue-server.com",
"HUE_USERNAME": "your_username",
"HUE_PASSWORD": "your_password"
}
}
}
}
```
### Development Mode
Test the server interactively using the MCP inspector:
```bash
uv run mcp dev src/hue_mcp_server/server.py
```
This opens an interactive interface where you can test tools and see requests/responses in real-time.
### Direct Command Line Execution
You can also run the server directly:
```bash
# Using the installed script (after uv sync)
uv run hue-mcp-server
# Or via Python module
uv run python -m hue_mcp_server.server
```
## Available Tools
### SQL Query Tools
#### `hue_execute_query`
Execute a SQL query and return results directly.
```
Arguments:
- statement: SQL statement to execute
- dialect: 'hive', 'sparksql', or 'impala' (default: 'hive')
- timeout: Max wait time in seconds (default: 300)
- batch_size: Rows per batch (default: 1000)
```
#### `hue_run_query_to_csv`
Execute a query and save results to a CSV file.
```
Arguments:
- statement: SQL statement to execute
- filename: Output CSV filename (default: 'results.csv')
- dialect: SQL dialect (default: 'hive')
- batch_size: Rows per batch (default: 1000)
```
#### `hue_export_and_download`
Execute INSERT OVERWRITE DIRECTORY and download resulting files.
```
Arguments:
- statement: SQL with INSERT OVERWRITE DIRECTORY
- hdfs_directory: HDFS output directory
- local_directory: Local download directory (default: '.')
- dialect: SQL dialect (default: 'hive')
- file_pattern: Regex to filter files (optional)
- timeout: Max wait time (default: 300)
```
### HDFS File Tools
#### `hue_list_directory`
List contents of an HDFS directory.
```
Arguments:
- directory_path: HDFS path (e.g., '/user/data')
- page_size: Max items to return (default: 1000)
```
#### `hue_check_directory_exists`
Check if an HDFS directory exists.
```
Arguments:
- directory_path: HDFS path to check
```
#### `hue_download_file`
Download a single file from HDFS.
```
Arguments:
- remote_path: Full HDFS file path
- local_filename: Local filename (optional)
```
#### `hue_download_directory`
Download all files from an HDFS directory.
```
Arguments:
- directory_path: HDFS directory path
- local_directory: Local directory (default: '.')
- file_pattern: Regex to filter files (optional)
```
#### `hue_upload_file`
Upload a local file to HDFS.
```
Arguments:
- local_file_path: Path to local file
- hdfs_destination: HDFS destination directory
```
## How It Works
### The Model Context Protocol (MCP)
MCP is an open protocol that standardizes how AI assistants communicate with external tools and data sources. Think of it as a universal adapter that lets AI assistants "plug into" different services.
**Key components:**
- **MCP Server** (this project): Exposes tools and capabilities
- **MCP Client** (VS Code/Claude Desktop): Consumes tools and presents them to the AI
- **Protocol**: Defines how they communicate
### Architecture Flow
```
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ GitHub │ MCP │ Hue MCP │ REST │ Hadoop Hue │
│ Copilot │◄──────►│ Server │◄──────►│ Server │
│ (VS Code) │Protocol│ (This Project) │ API │ │
└─────────────────┘ └──────────────────┘ └─────────────────┘
│
▼
┌──────────────────┐
│ HueClientRest │
│ Library │
└──────────────────┘
```
1. **User asks Copilot** to query Hue data
2. **Copilot recognizes** the request requires Hue MCP tools
3. **MCP Server receives** the request and translates it to Hue REST API calls
4. **HueClientRest library** handles authentication and API communication
5. **Results flow back** through the chain to the user
### Dependency Details
#### Astral uv (Package Manager)
**What it is:** A next-generation Python package and project manager written in Rust.
**Why we use it:**
- **Speed**: 10-100x faster than pip
- **Better dependency resolution**: Handles complex dependencies more reliably
- **Unified tool**: Combines pip, pip-tools, pipx, poetry, pyenv functionality
- **Reproducible environments**: Lock files ensure consistent installs
- **Cross-platform**: Works seamlessly on Windows, macOS, and Linux
**Key commands:**
- `uv sync` - Install/update dependencies
- `uv add <package>` - Add a new dependency
- `uv run <command>` - Run commands in the virtual environment
- `uv pip install <package>` - Use like pip if needed
**Learn more:** [https://docs.astral.sh/uv/](https://docs.astral.sh/uv/)
#### mcp[cli] (Python SDK)
**What it is:** The official Python SDK for building MCP servers.
**Key features:**
- **FastMCP framework**: Simplified server creation with decorators
- **Type validation**: Pydantic integration for request/response validation
- **CLI tools**: `mcp dev` for testing, `mcp install` for setup
- **Async support**: Built on asyncio for efficient I/O
- **SSE transport**: Server-sent events for real-time communication
**In this project:**
```python
from mcp.server.fastmcp import FastMCP
mcp = FastMCP("Hue MCP Server")
@mcp.tool()
def hue_execute_query(statement: str, dialect: str = "hive"):
"""Execute SQL query on Hue"""
# Implementation
```
#### hueclientrest (Hue Client Library)
**What it is:** Python client for Hadoop Hue REST API.
**Capabilities:**
- SQL query execution (Hive, SparkSQL, Impala)
- HDFS file operations
- Session management
- Authentication handling
- Error handling and retries
**In this project:**
```python
from hueclientrest import HueClientREST
client = HueClientREST(host, username, password)
client.login()
result = client.execute_query(statement, dialect)
```
#### pydantic (Data Validation)
**What it is:** Data validation library using Python type hints.
**Why we use it:**
- **Type safety**: Validates tool inputs/outputs at runtime
- **Auto-documentation**: Generates schemas from type hints
- **Error messages**: Clear validation errors for debugging
- **JSON schema**: Automatic schema generation for MCP
**In this project:**
```python
from pydantic import BaseModel, Field
class QueryResult(BaseModel):
rows: List[dict]
columns: List[str]
row_count: int
```
## Example Usage Scenarios
### Scenario 1: Execute a Hive Query
**In VS Code with Copilot:**
```
You: "Execute a Hive query to show the first 10 tables"
Copilot: [Uses hue_execute_query tool]
Result: Returns table list from your Hue server
```
**Query executed:**
```sql
SELECT database_name, table_name
FROM information_schema.tables
LIMIT 10
```
### Scenario 2: List HDFS Files
**In VS Code with Copilot:**
```
You: "List all files in /user/hive/warehouse directory"
Copilot: [Uses hue_list_directory tool]
Result: Shows file names, sizes, and permissions
```
### Scenario 3: Export Data to CSV
**In VS Code with Copilot:**
```
You: "Query the sales table for 2024 and save to CSV"
Copilot: [Uses hue_run_query_to_csv tool]
Result: Creates sales_2024.csv with query results
```
**Query executed:**
```sql
SELECT * FROM sales WHERE year = 2024
```
### Scenario 4: Complex Data Pipeline
**In VS Code with Copilot:**
```
You: "Check if /user/data/processed exists, if not list /user/data,
then download all CSV files from there"
Copilot:
1. [Uses hue_check_directory_exists]
2. [Uses hue_list_directory]
3. [Uses hue_download_directory with file_pattern=".*\.csv$"]
Result: Downloads all CSV files to local directory
```
## Development
### Setting Up for Development
```bash
# Clone and install
git clone <your-repo-url>
cd hueclientrest-mpc
uv sync
# Install development dependencies
uv sync --dev
```
### Running Tests
```bash
# Run all tests
uv run pytest
# Run with coverage
uv run pytest --cov=hue_mcp_server
# Run specific test file
uv run pytest tests/test_server.py
```
### Adding New Tools
To add a new MCP tool:
1. Define the tool function in `server.py`:
```python
@mcp.tool()
def hue_new_feature(param: str) -> dict:
"""Description of what this tool does."""
client = get_client()
result = client.some_operation(param)
return {"status": "success", "data": result}
```
2. The `@mcp.tool()` decorator automatically:
- Registers the tool with the MCP server
- Generates JSON schema from type hints
- Validates inputs using Pydantic
- Handles errors and responses
3. Test your tool:
```bash
uv run mcp dev src/hue_mcp_server/server.py
```
### Debugging
**Enable verbose logging:**
```python
import logging
logging.basicConfig(level=logging.DEBUG)
```
**Test MCP server directly:**
```bash
# Interactive testing
uv run mcp dev src/hue_mcp_server/server.py
# Check server can start
uv run python -m hue_mcp_server.server
```
**VS Code debugging:**
1. Check Output panel: View > Output > GitHub Copilot
2. Look for MCP server connection messages
3. Check for authentication or network errors
### Project Structure
```
hueclientrest-mpc/
├── .venv/ # Virtual environment (created by uv)
├── pyproject.toml # Project metadata and dependencies
├── README.md # This comprehensive guide
├── .env.example # Example environment variables
├── .gitignore # Git ignore patterns
└── src/
└── hue_mcp_server/
├── __init__.py # Package initialization
└── server.py # MCP server implementation
├── Server setup and configuration
├── Tool definitions (@mcp.tool decorators)
├── Hue client wrapper functions
└── Main entry point
```
### Dependencies Management
**View installed packages:**
```bash
uv pip list
```
**Add a new dependency:**
```bash
uv add <package-name>
```
**Update dependencies:**
```bash
uv sync --upgrade
```
**Remove a dependency:**
```bash
uv remove <package-name>
```
## Security Considerations
### Credential Management
**Best Practices:**
1. **Never commit credentials** to version control
2. **Use environment variables** or secure vaults
3. **Rotate passwords** regularly
4. **Use .env files** for local development only
5. **Use secrets management** (Azure Key Vault, AWS Secrets Manager) in production
### SSL/TLS Configuration
For production environments:
```bash
# Always verify SSL certificates
HUE_VERIFY_SSL=true
HUE_SSL_WARNINGS=false
```
For development/testing (self-signed certificates):
```bash
# Only for development!
HUE_VERIFY_SSL=false
HUE_SSL_WARNINGS=false
```
### Network Security
- Ensure Hue server is accessible from your development machine
- Check firewall rules if connection fails
- Use VPN if required by your organization
- Keep authentication tokens secure
## Troubleshooting Common Issues
### Issue: uv command not found
**Solution:**
```bash
# Windows PowerShell
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
# Then restart terminal or add to PATH
```
### Issue: Python version mismatch
**Error:** `requires-python = ">=3.10"` but you have Python 3.9
**Solution:**
```bash
# Install Python 3.10+ from python.org
# Or use uv to manage Python versions
uv python install 3.11
uv venv --python 3.11
```
### Issue: MCP server not loading in VS Code
**Checklist:**
- [ ] uv is installed and in PATH (`uv --version`)
- [ ] Project dependencies installed (`uv sync`)
- [ ] mcp.json path is correct for your OS
- [ ] Project path in mcp.json uses proper escaping
- [ ] VS Code completely restarted (all windows closed)
- [ ] GitHub Copilot extension is enabled
- [ ] Active Copilot subscription
### Issue: Authentication failures
**Error:** "Authentication failed" or "401 Unauthorized"
**Solution:**
1. Verify credentials are correct
2. Check if Hue server URL is accessible
3. Test login directly in browser
4. Check for special characters in password (may need escaping)
5. Verify user has necessary permissions in Hue
### Issue: Query timeouts
**Error:** "Query execution timeout"
**Solution:**
```python
# Increase timeout when calling tools
hue_execute_query(
statement="SELECT * FROM large_table",
timeout=600 # 10 minutes instead of default 5
)
```
### Issue: HDFS file not found
**Error:** "File or directory not found"
**Solution:**
1. Verify path is absolute (starts with `/`)
2. Check permissions on HDFS
3. Use `hue_list_directory` to browse available paths
4. Verify user has read/write permissions
## Performance Tips
### Query Optimization
1. **Use batch_size** for large result sets:
```python
hue_execute_query(statement="...", batch_size=5000)
```
2. **Use LIMIT** in queries when exploring:
```sql
SELECT * FROM large_table LIMIT 1000
```
3. **Export large datasets** directly to HDFS:
```sql
INSERT OVERWRITE DIRECTORY '/tmp/export'
SELECT * FROM large_table
```
Then use `hue_export_and_download` to retrieve files.
### HDFS Operations
1. **Download specific files** with patterns:
```python
hue_download_directory(
directory_path="/user/data",
file_pattern=".*\\.csv$" # Only CSV files
)
```
2. **Use streaming** for large file transfers
3. **Batch uploads** when possible
## FAQ
### Q: Can I use this with Claude Desktop?
**A:** Yes! See the "Claude Desktop Integration" section for configuration details.
### Q: Does this work on Windows?
**A:** Yes, fully supported on Windows, macOS, and Linux.
### Q: What Hue versions are supported?
**A:** Any version with REST API support. Tested with Hue 4.x and newer.
### Q: Can multiple users share one MCP server?
**A:** Each user should run their own MCP server instance with their own credentials.
### Q: How do I update to the latest version?
**A:**
```bash
git pull
uv sync --upgrade
```
### Q: Is my password secure?
**A:** Credentials are stored in environment variables or mcp.json. Keep these files secure and never commit them to version control.
## Resources
### Documentation
- [Model Context Protocol Specification](https://modelcontextprotocol.io/)
- [MCP Python SDK Documentation](https://github.com/modelcontextprotocol/python-sdk)
- [Astral uv Documentation](https://docs.astral.sh/uv/)
- [HueClientRest Repository](https://github.com/SpanishST/HueClientRest)
- [Pydantic Documentation](https://docs.pydantic.dev/)
### Community
- [MCP Discord Server](https://discord.gg/mcp)
- [GitHub Discussions](https://github.com/modelcontextprotocol/python-sdk/discussions)
### Related Projects
- [MCP Servers Collection](https://github.com/modelcontextprotocol/servers)
- [Awesome MCP](https://github.com/punkpeye/awesome-mcp)
## Contributing
Contributions are welcome! Please feel free to submit issues or pull requests.
### Development Setup
```bash
git clone <your-repo-url>
cd hueclientrest-mpc
uv sync --dev
```
### Running Tests
```bash
uv run pytest
uv run pytest --cov=hue_mcp_server
```
### Code Style
This project uses:
- Black for code formatting
- isort for import sorting
- mypy for type checking
## Changelog
### v0.1.0 (Current)
- Initial release
- SQL query execution (Hive, SparkSQL, Impala)
- HDFS file operations
- CSV export functionality
- VS Code and Claude Desktop integration
## License
MIT License - See LICENSE file for details
## Credits
- **[HueClientRest](https://github.com/SpanishST/HueClientRest)** - The underlying Python client for Hue REST API
- **[Model Context Protocol](https://modelcontextprotocol.io/)** - Open standard for AI-tool integration
- **[Astral](https://astral.sh/)** - Creators of uv package manager
- **[Anthropic](https://www.anthropic.com/)** - MCP specification and implementation
## Support
For issues, questions, or feature requests:
1. Check the [Troubleshooting](#troubleshooting-common-issues) section
2. Search existing GitHub issues
3. Create a new issue with detailed information
---
**Happy querying! 🚀**
