# DP-MCP Server Deployment Guide
This guide covers various deployment scenarios for the DP-MCP server, from development to production environments.
## Table of Contents
- [Prerequisites](#prerequisites)
- [Development Deployment](#development-deployment)
- [Production Deployment](#production-deployment)
- [Docker Deployment](#docker-deployment)
- [Cloud Deployment](#cloud-deployment)
- [Monitoring and Logging](#monitoring-and-logging)
- [Security Hardening](#security-hardening)
- [Backup and Recovery](#backup-and-recovery)
- [Troubleshooting](#troubleshooting)
## Prerequisites
### System Requirements
**Minimum Requirements:**
- CPU: 2 cores
- RAM: 4GB
- Storage: 10GB available space
- Network: Stable internet connection
**Recommended for Production:**
- CPU: 4+ cores
- RAM: 8GB+
- Storage: 50GB+ SSD
- Network: High-bandwidth connection
### Software Dependencies
**Required:**
- Python 3.10 or higher
- Git
- PostgreSQL client tools (psql)
- MinIO client (mc) - optional but recommended
**For Docker Deployment:**
- Docker Engine 20.10+
- Docker Compose 2.0+
**For Production:**
- Systemd (Linux) or equivalent service manager
- Nginx or Apache (reverse proxy)
- SSL certificates (Let's Encrypt recommended)
## Development Deployment
### Quick Setup
```bash
# Clone repository
git clone <repository-url>
cd da.dp-mcp
# Run setup script
chmod +x setup.sh
./setup.sh
# Start development services
docker-compose up -d
# Launch server
uv run python src/dp_mcp/server.py --debug
```
### Development Configuration
Create `.env` file for development:
```bash
# Development PostgreSQL
POSTGRES_HOST=localhost
POSTGRES_PORT=5432
POSTGRES_USER=dp_mcp_user
POSTGRES_PASSWORD=dp_mcp_password
POSTGRES_DATABASE=dp_mcp_dev
POSTGRES_SSL_MODE=prefer
# Development MinIO
MINIO_ENDPOINT=localhost:9000
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=minioadmin123
MINIO_SECURE=false
MINIO_DEFAULT_BUCKET=default-bucket
# Development Server
MCP_SERVER_HOST=127.0.0.1
MCP_SERVER_PORT=8888
DEBUG=true
```
### Accessing Development Services
- **MCP Server**: http://localhost:8888/mcp/
- **MinIO Console**: http://localhost:9001 (minioadmin/minioadmin123)
- **PostgreSQL**: localhost:5432 (dp_mcp_user/dp_mcp_password)
## Production Deployment
### 1. Server Preparation
```bash
# Update system
sudo apt update && sudo apt upgrade -y
# Install required packages
sudo apt install -y python3.10 python3.10-venv git nginx certbot
# Create deployment user
sudo useradd -m -s /bin/bash dp-mcp
sudo usermod -aG sudo dp-mcp
# Switch to deployment user
sudo su - dp-mcp
```
### 2. Application Setup
```bash
# Clone repository
git clone <repository-url> /opt/dp-mcp
cd /opt/dp-mcp
# Install uv package manager
curl -LsSf https://astral.sh/uv/install.sh | sh
source ~/.bashrc
# Install dependencies
uv sync --frozen
# Install application
uv pip install -e .
```
### 3. Production Configuration
Create production `.env` file:
```bash
# Production PostgreSQL
POSTGRES_HOST=your-db-host.example.com
POSTGRES_PORT=5432
POSTGRES_USER=dp_mcp_prod
POSTGRES_PASSWORD=secure_random_password_123
POSTGRES_DATABASE=dp_mcp_production
POSTGRES_SSL_MODE=require
# Production MinIO
MINIO_ENDPOINT=your-minio-host.example.com:9000
MINIO_ACCESS_KEY=prod_access_key_123
MINIO_SECRET_KEY=secure_secret_key_456
MINIO_SECURE=true
MINIO_DEFAULT_BUCKET=production-bucket
# Production Server
MCP_SERVER_HOST=127.0.0.1
MCP_SERVER_PORT=8888
DEBUG=false
```
### 4. Systemd Service Setup
Create `/etc/systemd/system/dp-mcp.service`:
```ini
[Unit]
Description=DP-MCP Server
After=network.target postgresql.service
Wants=postgresql.service
[Service]
Type=simple
User=dp-mcp
Group=dp-mcp
WorkingDirectory=/opt/dp-mcp
Environment=PATH=/home/dp-mcp/.local/bin:/opt/dp-mcp/.venv/bin
EnvironmentFile=/opt/dp-mcp/.env
ExecStart=/opt/dp-mcp/.venv/bin/python src/dp_mcp/server.py
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=dp-mcp
# Security settings
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/dp-mcp
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
[Install]
WantedBy=multi-user.target
```
### 5. Start Services
```bash
# Reload systemd
sudo systemctl daemon-reload
# Enable and start service
sudo systemctl enable dp-mcp
sudo systemctl start dp-mcp
# Check status
sudo systemctl status dp-mcp
```
### 6. Reverse Proxy Setup (Nginx)
Create `/etc/nginx/sites-available/dp-mcp`:
```nginx
server {
listen 80;
server_name your-server.example.com;
return 301 https://$server_name$request_uri;
}
server {
listen 443 ssl http2;
server_name your-server.example.com;
# SSL configuration
ssl_certificate /etc/letsencrypt/live/your-server.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/your-server.example.com/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256;
ssl_prefer_server_ciphers off;
# Security headers
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
add_header X-XSS-Protection "1; mode=block";
add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";
# MCP server proxy
location /mcp/ {
proxy_pass http://127.0.0.1:8888/mcp/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# SSE support
proxy_set_header Connection '';
proxy_http_version 1.1;
chunked_transfer_encoding off;
proxy_buffering off;
proxy_cache off;
}
# Health check endpoint
location /health {
access_log off;
return 200 "healthy\n";
add_header Content-Type text/plain;
}
}
```
Enable the site:
```bash
sudo ln -s /etc/nginx/sites-available/dp-mcp /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx
```
### 7. SSL Certificate Setup
```bash
# Install Certbot
sudo apt install certbot python3-certbot-nginx
# Obtain SSL certificate
sudo certbot --nginx -d your-server.example.com
# Test renewal
sudo certbot renew --dry-run
```
## Docker Deployment
### Production Dockerfile
Create `Dockerfile.prod`:
```dockerfile
FROM python:3.11-slim
# Set environment variables
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1
# Install system dependencies
RUN apt-get update && apt-get install -y \
gcc \
postgresql-client \
curl \
&& rm -rf /var/lib/apt/lists/*
# Create app user
RUN useradd --create-home --shell /bin/bash dp-mcp
# Set work directory
WORKDIR /app
# Install uv
RUN pip install uv
# Copy dependency files
COPY pyproject.toml ./
COPY uv.lock ./
# Install dependencies
RUN uv sync --frozen --no-dev
# Copy application code
COPY --chown=dp-mcp:dp-mcp . .
# Switch to app user
USER dp-mcp
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD curl -f http://localhost:8888/health || exit 1
# Expose port
EXPOSE 8888
# Run application
CMD ["uv", "run", "python", "src/dp_mcp/server.py", "--host", "0.0.0.0", "--port", "8888"]
```
### Docker Compose Production
Create `docker-compose.prod.yml`:
```yaml
version: '3.8'
services:
dp-mcp:
build:
context: .
dockerfile: Dockerfile.prod
container_name: dp-mcp-server
restart: unless-stopped
environment:
- POSTGRES_HOST=${POSTGRES_HOST}
- POSTGRES_PORT=${POSTGRES_PORT}
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
- POSTGRES_DATABASE=${POSTGRES_DATABASE}
- POSTGRES_SSL_MODE=${POSTGRES_SSL_MODE}
- MINIO_ENDPOINT=${MINIO_ENDPOINT}
- MINIO_ACCESS_KEY=${MINIO_ACCESS_KEY}
- MINIO_SECRET_KEY=${MINIO_SECRET_KEY}
- MINIO_SECURE=${MINIO_SECURE}
- MCP_SERVER_HOST=0.0.0.0
- MCP_SERVER_PORT=8888
- DEBUG=false
ports:
- "8888:8888"
volumes:
- ./logs:/app/logs
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8888/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
nginx:
image: nginx:alpine
container_name: dp-mcp-nginx
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
- ./ssl:/etc/ssl/certs:ro
depends_on:
- dp-mcp
```
### Deploy with Docker
```bash
# Build and start
docker-compose -f docker-compose.prod.yml up -d
# Check logs
docker-compose -f docker-compose.prod.yml logs -f dp-mcp
# Scale if needed
docker-compose -f docker-compose.prod.yml up -d --scale dp-mcp=3
```
## Cloud Deployment
### AWS Deployment
#### ECS Fargate Setup
1. **Create ECS Cluster:**
```bash
aws ecs create-cluster --cluster-name dp-mcp-cluster
```
2. **Create Task Definition:**
```json
{
"family": "dp-mcp-task",
"networkMode": "awsvpc",
"requiresCompatibilities": ["FARGATE"],
"cpu": "256",
"memory": "512",
"executionRoleArn": "arn:aws:iam::ACCOUNT:role/ecsTaskExecutionRole",
"containerDefinitions": [
{
"name": "dp-mcp",
"image": "your-account.dkr.ecr.region.amazonaws.com/dp-mcp:latest",
"portMappings": [
{
"containerPort": 8888,
"protocol": "tcp"
}
],
"environment": [
{"name": "POSTGRES_HOST", "value": "your-rds-endpoint"},
{"name": "MINIO_ENDPOINT", "value": "your-s3-endpoint"}
],
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "/ecs/dp-mcp",
"awslogs-region": "us-east-1",
"awslogs-stream-prefix": "ecs"
}
}
}
]
}
```
3. **Create ECS Service:**
```bash
aws ecs create-service \
--cluster dp-mcp-cluster \
--service-name dp-mcp-service \
--task-definition dp-mcp-task \
--desired-count 2 \
--launch-type FARGATE \
--network-configuration "awsvpcConfiguration={subnets=[subnet-12345],securityGroups=[sg-12345],assignPublicIp=ENABLED}"
```
### Google Cloud Platform
#### Cloud Run Deployment
```bash
# Build and push to Container Registry
gcloud builds submit --tag gcr.io/PROJECT-ID/dp-mcp
# Deploy to Cloud Run
gcloud run deploy dp-mcp \
--image gcr.io/PROJECT-ID/dp-mcp \
--platform managed \
--region us-central1 \
--allow-unauthenticated \
--port 8888 \
--set-env-vars POSTGRES_HOST=your-postgres-host,MINIO_ENDPOINT=your-storage-endpoint
```
### Azure Container Instances
```bash
# Create resource group
az group create --name dp-mcp-rg --location eastus
# Deploy container
az container create \
--resource-group dp-mcp-rg \
--name dp-mcp-container \
--image your-registry/dp-mcp:latest \
--dns-name-label dp-mcp-unique \
--ports 8888 \
--environment-variables \
POSTGRES_HOST=your-postgres-host \
MINIO_ENDPOINT=your-storage-endpoint
```
## Monitoring and Logging
### Application Monitoring
#### Health Check Endpoint
Add to `src/dp_mcp/server.py`:
```python
@mcp.tool()
async def health_check() -> str:
"""Health check endpoint for monitoring."""
try:
# Test database connection
conn = get_db_connection()
conn.close()
# Test MinIO connection
client = get_minio_client()
list(client.list_buckets())
return "healthy"
except Exception as e:
return f"unhealthy: {str(e)}"
```
#### Prometheus Metrics
Install prometheus client:
```bash
uv add prometheus-client
```
Add metrics to server:
```python
from prometheus_client import Counter, Histogram, generate_latest
# Metrics
REQUEST_COUNT = Counter('dp_mcp_requests_total', 'Total requests', ['tool', 'status'])
REQUEST_DURATION = Histogram('dp_mcp_request_duration_seconds', 'Request duration', ['tool'])
@mcp.tool()
async def metrics() -> str:
"""Prometheus metrics endpoint."""
return generate_latest().decode()
```
### Log Configuration
Create `logging.conf`:
```ini
[loggers]
keys=root,dp_mcp
[handlers]
keys=consoleHandler,fileHandler
[formatters]
keys=standardFormatter
[logger_root]
level=INFO
handlers=consoleHandler
[logger_dp_mcp]
level=INFO
handlers=consoleHandler,fileHandler
qualname=dp_mcp
propagate=0
[handler_consoleHandler]
class=StreamHandler
level=INFO
formatter=standardFormatter
args=(sys.stdout,)
[handler_fileHandler]
class=handlers.RotatingFileHandler
level=INFO
formatter=standardFormatter
args=('logs/dp-mcp.log', 'a', 10485760, 5)
[formatter_standardFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=%Y-%m-%d %H:%M:%S
```
### Grafana Dashboard
Create dashboard JSON for visualization:
```json
{
"dashboard": {
"title": "DP-MCP Server Monitoring",
"panels": [
{
"title": "Request Rate",
"type": "graph",
"targets": [
{
"expr": "rate(dp_mcp_requests_total[5m])",
"legendFormat": "{{tool}} - {{status}}"
}
]
},
{
"title": "Response Time",
"type": "graph",
"targets": [
{
"expr": "histogram_quantile(0.95, dp_mcp_request_duration_seconds_bucket)",
"legendFormat": "95th percentile"
}
]
}
]
}
}
```
## Security Hardening
### Application Security
1. **Environment Variables:**
```bash
# Never expose sensitive data in logs
export POSTGRES_PASSWORD="$(openssl rand -base64 32)"
export MINIO_SECRET_KEY="$(openssl rand -base64 32)"
```
2. **Database Security:**
```sql
-- Create read-only user for MCP server
CREATE USER dp_mcp_readonly WITH PASSWORD 'secure_password';
GRANT CONNECT ON DATABASE production_db TO dp_mcp_readonly;
GRANT USAGE ON SCHEMA public TO dp_mcp_readonly;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO dp_mcp_readonly;
```
3. **MinIO Security:**
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::allowed-bucket/*",
"arn:aws:s3:::allowed-bucket"
]
}
]
}
```
### Network Security
1. **Firewall Rules:**
```bash
# Allow only necessary ports
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
```
2. **Fail2Ban Configuration:**
```ini
[dp-mcp]
enabled = true
port = 8888
filter = dp-mcp
logpath = /var/log/dp-mcp.log
maxretry = 5
bantime = 3600
```
## Backup and Recovery
### Database Backup
```bash
#!/bin/bash
# backup-db.sh
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_DIR="/opt/backups/postgres"
BACKUP_FILE="$BACKUP_DIR/dp_mcp_backup_$DATE.sql"
mkdir -p $BACKUP_DIR
pg_dump -h $POSTGRES_HOST -U $POSTGRES_USER -d $POSTGRES_DATABASE > $BACKUP_FILE
# Upload to MinIO
mc cp $BACKUP_FILE minio/backups/postgres/
# Keep only last 30 days
find $BACKUP_DIR -name "*.sql" -mtime +30 -delete
```
### Application Backup
```bash
#!/bin/bash
# backup-app.sh
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_DIR="/opt/backups/application"
mkdir -p $BACKUP_DIR
# Backup configuration
tar -czf "$BACKUP_DIR/dp_mcp_config_$DATE.tar.gz" \
/opt/dp-mcp/.env \
/opt/dp-mcp/pyproject.toml \
/etc/systemd/system/dp-mcp.service \
/etc/nginx/sites-available/dp-mcp
# Upload to MinIO
mc cp "$BACKUP_DIR/dp_mcp_config_$DATE.tar.gz" minio/backups/config/
```
### Automated Backup with Cron
```bash
# Add to crontab
0 2 * * * /opt/scripts/backup-db.sh
0 3 * * * /opt/scripts/backup-app.sh
```
## Troubleshooting
### Common Issues
1. **Service Won't Start:**
```bash
# Check service status
sudo systemctl status dp-mcp
# Check logs
sudo journalctl -u dp-mcp -f
# Verify configuration
uv run python src/dp_mcp/utils/config.py
```
2. **High Memory Usage:**
```bash
# Monitor memory usage
htop
# Check for memory leaks
python -m tracemalloc src/dp_mcp/server.py
# Adjust query limits
export DEFAULT_QUERY_LIMIT=1000
```
3. **Slow Database Queries:**
```sql
-- Enable query logging
ALTER SYSTEM SET log_statement = 'all';
ALTER SYSTEM SET log_min_duration_statement = 1000;
SELECT pg_reload_conf();
-- Check slow queries
SELECT query, mean_time, calls
FROM pg_stat_statements
ORDER BY mean_time DESC
LIMIT 10;
```
### Performance Optimization
1. **Database Connection Pooling:**
```python
# Add to postgres_tools.py
import psycopg2.pool
connection_pool = psycopg2.pool.ThreadedConnectionPool(
1, 20, # min and max connections
host=os.getenv("POSTGRES_HOST"),
port=int(os.getenv("POSTGRES_PORT", 5432)),
user=os.getenv("POSTGRES_USER"),
password=os.getenv("POSTGRES_PASSWORD"),
database=os.getenv("POSTGRES_DATABASE")
)
```
2. **Caching Layer:**
```python
import redis
cache = redis.Redis(host='localhost', port=6379, db=0)
@functools.lru_cache(maxsize=128)
def cached_query(query_hash):
# Implement query result caching
pass
```
3. **Load Balancing:**
```nginx
upstream dp_mcp_backend {
server 127.0.0.1:8888;
server 127.0.0.1:8889;
server 127.0.0.1:8890;
}
server {
location /mcp/ {
proxy_pass http://dp_mcp_backend;
}
}
```
For additional support, consult the main [README.md](../README.md) or open an issue in the project repository.
