How do I use Azure ML MCP Server?

1. Click on "Install Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@Azure ML MCP Server predict sales for next month using my forecasting endpoint" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

Azure ML MCP Server on Azure Container Apps

Deploy a Model Context Protocol (MCP) server to Azure Container Apps that exposes Azure ML endpoints as tools for AI agents.

🎯 What This Does

This project creates an MCP server that:

Runs on Azure Container Apps (serverless, scales to zero)
Exposes Azure ML managed online endpoints as MCP tools
Works with Azure AI Foundry agents, Copilot Studio, and any MCP-compatible client

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│  AI Foundry     │────▶│  MCP Server     │────▶│  Azure ML       │
│  Agent          │ MCP │  (Container App)│HTTP │  Endpoint       │
└─────────────────┘     └─────────────────┘     └─────────────────┘

📋 Prerequisites

Azure Subscription with permissions to create resources
Azure ML Managed Online Endpoint deployed and running
Azure CLI installed (Install Guide)
Docker installed for local development and building images
Python 3.11+ for local testing

🚀 Quick Start

1. Clone and Configure

# Clone the repository
git clone <your-repo-url>
cd <your-repo-folder>

# Copy environment template
cp .env.sample .env   # Linux/macOS
copy .env.sample .env  # Windows CMD
Copy-Item .env.sample .env  # Windows PowerShell

# Edit .env with your Azure ML endpoint details
# Get these from Azure ML Studio > Endpoints > Your endpoint > Consume tab

2. Test Locally

# Create virtual environment
python -m venv .venv
.venv\Scripts\activate  # Windows
# source .venv/bin/activate  # Linux/Mac

# Install dependencies
pip install -r requirements.txt

# Run the server
python server.py

Test with curl:

curl -X POST http://localhost:8080/mcp/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"tools/list","id":1}'

3. Deploy to Azure

Bash (Linux/macOS/WSL)

# Login to Azure
az login

# Create resource group
az group create --name rg-mcp-server --location eastus

# Update parameters with your Azure ML credentials
# Edit infra/main.parameters.bicepparam

# Deploy infrastructure
az deployment group create \
  --resource-group rg-mcp-server \
  --template-file infra/main.bicep \
  --parameters infra/main.parameters.bicepparam

# Get the ACR name from the output
ACR_NAME=$(az deployment group show -g rg-mcp-server -n main --query properties.outputs.acrName.value -o tsv)

# Build and push Docker image
az acr login --name $ACR_NAME
docker build -t $ACR_NAME.azurecr.io/mcp-server:latest .
docker push $ACR_NAME.azurecr.io/mcp-server:latest

# Restart the container app to pull the new image
az containerapp update \
  --name mcp-server \
  --resource-group rg-mcp-server

PowerShell (Windows)

# Login to Azure
az login

# Create resource group
az group create --name rg-mcp-server --location eastus

# Update parameters with your Azure ML credentials
# Edit infra/main.parameters.bicepparam

# Deploy infrastructure
az deployment group create `
  --resource-group rg-mcp-server `
  --template-file infra/main.bicep `
  --parameters infra/main.parameters.bicepparam

# Get the ACR name from the output
$ACR_NAME = az deployment group show -g rg-mcp-server -n main --query properties.outputs.acrName.value -o tsv

# Build and push Docker image
az acr login --name $ACR_NAME
docker build -t "$ACR_NAME.azurecr.io/mcp-server:latest" .
docker push "$ACR_NAME.azurecr.io/mcp-server:latest"

# Restart the container app to pull the new image
az containerapp update `
  --name mcp-server `
  --resource-group rg-mcp-server

4. Get Your MCP Endpoint

az deployment group show \
  --resource-group rg-mcp-server \
  --name main \
  --query properties.outputs.mcpEndpoint.value -o tsv

Your MCP endpoint will be: https://<your-app>.azurecontainerapps.io/mcp/mcp

🤖 Connect to Azure AI Foundry

Go to Azure AI Foundry
Navigate to your project > Agent > Tools
Click + New tool > MCP Tool
Configure:
- Name: AzureMLScoring (or your preferred name)
- Server URL: https://<your-app>.azurecontainerapps.io/mcp/mcp
- Auth Type: None (the app uses managed secrets internally)
Save and test the tool in your agent

📁 Project Structure

├── server.py              # MCP server with Azure ML tool
├── requirements.txt       # Python dependencies
├── Dockerfile            # Container image definition
├── .env.sample           # Environment variables template
├── .gitignore            # Git ignore rules
├── .dockerignore         # Docker build exclusions
└── infra/
    ├── main.bicep              # Azure infrastructure definition
    └── main.parameters.bicepparam  # Deployment parameters

🔧 Customizing the Tool

⚠️ Important: The example code is configured for an abitrary forecasting model. You must modify server.py to match your own Azure ML model's expected input schema (column names, data types) and output format.

Edit server.py to modify the MCP tool:

Change the function parameters to match your model's inputs
Update the DataFrame columns to match your model's expected schema
Modify the docstring to describe your tool accurately (this is what AI agents see)

@mcp.tool()
def invoke_azure_ml_endpoint(
    # Change these parameters to match your model's inputs
    your_param_1: float,
    your_param_2: str,
) -> float:
    """
    Update this docstring to describe your tool - AI agents use this to understand
    when and how to call your tool.
    """
    # Modify the DataFrame columns to match your model's expected schema
    df = pd.DataFrame(
        [[float(your_param_1), your_param_2]], 
        columns=["YourColumn1", "YourColumn2"]  # Change to your model's column names
    )
    
    # The payload structure may need adjustment - test your model in the 
    # Azure ML Studio 'Test' tab to see the expected format
    data = {"input_data": df.to_dict(orient='split')}
    # ... rest of the function

🐛 Troubleshooting

421 Misdirected Request Error

If you see this error in logs, ensure you have DNS rebinding protection disabled:

from mcp.server.fastmcp import FastMCP
from mcp.server.transport_security import TransportSecuritySettings

mcp = FastMCP(
    "your-server-name",
    stateless_http=True,
    transport_security=TransportSecuritySettings(enable_dns_rebinding_protection=False)
)

See: GitHub Issue #1798

Container Not Starting

Check logs:

az containerapp logs show \
  --name mcp-server \
  --resource-group rg-mcp-server \
  --follow

Azure ML Endpoint Errors

Verify your endpoint is accessible:

curl -X POST $AML_SCORE_URL \
  -H "Authorization: Bearer $AML_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input_data": {"columns": ["col1"], "data": [[1]]}}'

📚 Resources

⚠️ Production Considerations

This sample is designed for learning and prototyping. Before deploying to production, consider the following:

🔐 Authentication & Authorization

No endpoint authentication: The MCP endpoint is publicly accessible. For production, consider:
- Azure Container Apps authentication (Easy Auth)
- API key validation in your application code
- Azure API Management as a gateway
- OAuth 2.0 / Microsoft Entra ID integration

🌐 Network Security

Public ingress: The Container App is exposed to the internet. For enterprise scenarios, consider:
- VNet integration for private networking
- Private endpoints to restrict access
- Network Security Groups (NSGs) to control traffic
- Connecting to Azure ML endpoints via private endpoints

🔑 Secrets Management

Inline secrets: Secrets are stored directly in Container Apps configuration. For production:
- Use Azure Key Vault with managed identity
- Rotate secrets regularly
- Avoid storing secrets in parameter files (use Azure DevOps/GitHub secrets for CI/CD)

📊 Monitoring & Observability

Basic logging only: Consider adding:
- Application Insights for distributed tracing
- Custom metrics for model inference latency and error rates
- Alerting for failures and performance degradation

🏗️ Infrastructure

Single region: This sample deploys to one region. For high availability:
- Deploy to multiple regions with traffic manager
- Consider Azure Front Door for global load balancing
- Implement health probes and failover strategies

🛡️ Additional Enterprise Requirements

CORS: Currently allows all origins (*). Restrict to specific domains in production.
Rate limiting: No rate limiting configured. Consider API Management or application-level throttling.
Compliance: Ensure deployment meets your organization's compliance requirements (SOC 2, HIPAA, etc.)

📄 License

MIT License - See LICENSE for details.

Azure ML MCP Server