ADF Cost Intelligence MCP

Ask your AI assistant which ADF pipeline is bleeding your Azure budget, and exactly why.

An enterprise FinOps intelligence system for Azure Data Factory, exposed via the Model Context Protocol (MCP). Converts raw ADF consumption metrics (DIU-hours, vCore-hours, activity runs) into dollar estimates, detects waste, identifies cost spikes, and recommends specific fixes with quantified savings. Pricing is fetched live from the Azure Retail Prices API, not hardcoded constants.

Works with Claude Desktop, Claude Code, GitHub Copilot, Cursor, Windsurf, or any MCP-compatible client.

What You Can Ask

Which ADF pipeline cost the most this month?

Why did my ADF bill spike last week?

Are any of my pipelines running wastefully?

Give me your top recommendations to cut my ADF spend.

Break down the cost of my IngestSalesData pipeline.

Example response:

Your top 3 most expensive pipelines this month:

1. IngestSalesData - $142.30 (driver: DIU hours from Copy Activity)

2. TransformCustomerData - $87.50 (driver: Data Flow vCore hours)

3. LoadDailyReports - $34.20 - running hourly with zero rows processed

Recommendation: Switch LoadDailyReports to an event-based trigger. Estimated saving: $34.20/month ($410/year).

Related MCP server: Azure Usage MCP Server

The 5 Tools

Get Started in 5 Minutes

New user? Follow this path:

Step 1 - Clone and install         (~1 min)
Step 2 - Try with mock data        (~1 min)   no Azure needed
Step 3 - Connect your Azure ADF    (~10 min)  one-time setup
Step 4 - Add to your MCP client    (~2 min)

Jump to Quick Start below for the full steps.

Already have Python and an ADF instance? The shortest path to real data:

git clone https://github.com/harinarayn/adf-cost-intelligence-mcp
cd adf-cost-intelligence-mcp
pip install -r requirements.txt

# Create a service principal (one-time)
az ad sp create-for-rbac --name adf-cost-mcp-sp --role Reader \
  --scopes /subscriptions/{YOUR_SUB_ID}

# Grant it ADF read access (one-time)
az role assignment create --assignee {APP_ID} \
  --role "Data Factory Contributor" \
  --scope /subscriptions/{SUB}/resourceGroups/{RG}/providers/Microsoft.DataFactory/factories/{ADF}

# Configure credentials
cp .env.example .env
# Edit .env with your values

# Add to your MCP client config (see Connect Your MCP Client below)
# Then ask: "Which ADF pipeline cost the most this month?"

Security and Privacy

This server runs entirely in your own environment. No data ever leaves your infrastructure.

• Runs locally or in your own cloud, not our servers

• Only reads ADF run metadata (pipeline names, durations, status, DIU counts) - never your pipeline data or business content

• Service principal needs read-only access (see setup below)

• No telemetry, no callbacks, no third-party dependencies except the public Azure Retail Prices API (unauthenticated, read-only pricing data)

• Fully open source - every line of code is auditable

See SECURITY.md for the full security posture.

Prerequisites

Quick Start

Step 1 - Clone and install

git clone https://github.com/harinarayn/adf-cost-intelligence-mcp
cd adf-cost-intelligence-mcp
pip install -r requirements.txt
cp .env.example .env

Step 2 - Try it immediately with mock data (no Azure needed)

USE_MOCK_DATA=true python server.py

Open your MCP client and ask: "Which ADF pipeline cost the most this month?"

You'll see responses using realistic synthetic data covering 10 mock pipelines, including wasteful runs, cost spikes, and inactive pipelines.

Step 3 - Connect your real Azure ADF (~10 minutes, one-time)

3a. Login and find your subscription ID

az login
az account show --query id -o tsv

3b. Create a service principal with minimal permissions

az ad sp create-for-rbac \
  --name adf-cost-mcp-sp \
  --role Reader \
  --scopes /subscriptions/{YOUR_SUBSCRIPTION_ID}
# Save the output - you need: appId, password, tenant

3c. Grant read access to your ADF instance

az role assignment create \
  --assignee {APP_ID_FROM_ABOVE} \
  --role "Data Factory Contributor" \
  --scope /subscriptions/{SUB_ID}/resourceGroups/{RG}/providers/Microsoft.DataFactory/factories/{ADF_NAME}

Minimum permissions: The service principal only needs to read pipeline runs and activity runs. Data Factory Contributor is the closest built-in role. For tighter security, create a custom role with Microsoft.DataFactory/factories/read, Microsoft.DataFactory/factories/pipelineruns/read, and Microsoft.DataFactory/factories/activityruns/read.

3d. Enable per-pipeline billing in ADF (important)

1. Open ADF Studio

2. Go to Manage > Factory Settings

3. Set Billing by pipeline to ON and save

Without this, cost data is at factory level only, not per-pipeline.

Step 4 - Configure .env

AZURE_TENANT_ID=paste-from-step-3b
AZURE_CLIENT_ID=paste-appId-from-step-3b
AZURE_CLIENT_SECRET=paste-password-from-step-3b
AZURE_SUBSCRIPTION_ID=paste-from-step-3a
AZURE_RESOURCE_GROUP=your-resource-group-name
AZURE_DATA_FACTORY_NAME=your-adf-factory-name
USE_MOCK_DATA=false

Connect Your MCP Client

Claude Desktop

Mac/Linux: ~/.claude/claude_desktop_config.json Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "adf-cost-intelligence": {
      "command": "python",
      "args": ["/absolute/path/to/adf-cost-intelligence-mcp/server.py"],
      "env": {
        "AZURE_TENANT_ID": "your-tenant-id",
        "AZURE_CLIENT_ID": "your-app-id",
        "AZURE_CLIENT_SECRET": "your-secret",
        "AZURE_SUBSCRIPTION_ID": "your-subscription-id",
        "AZURE_RESOURCE_GROUP": "your-resource-group",
        "AZURE_DATA_FACTORY_NAME": "your-adf-name",
        "USE_MOCK_DATA": "false"
      }
    }
  }
}

Restart Claude Desktop. The tools appear automatically.

Claude Code (CLI)

claude mcp add adf-cost-intelligence python /path/to/server.py

Or add a .mcp.json to your project root (safe to commit, uses env var references):

{
  "servers": {
    "adf-cost-intelligence": {
      "type": "stdio",
      "command": "python",
      "args": ["server.py"],
      "env": {
        "AZURE_TENANT_ID": "${AZURE_TENANT_ID}",
        "AZURE_CLIENT_ID": "${AZURE_CLIENT_ID}",
        "AZURE_CLIENT_SECRET": "${AZURE_CLIENT_SECRET}",
        "AZURE_SUBSCRIPTION_ID": "${AZURE_SUBSCRIPTION_ID}",
        "AZURE_RESOURCE_GROUP": "${AZURE_RESOURCE_GROUP}",
        "AZURE_DATA_FACTORY_NAME": "${AZURE_DATA_FACTORY_NAME}",
        "USE_MOCK_DATA": "false"
      }
    }
  }
}

GitHub Copilot (VS Code)

Add to .vscode/mcp.json in your workspace:

{
  "servers": {
    "adf-cost-intelligence": {
      "type": "stdio",
      "command": "python",
      "args": ["/absolute/path/to/server.py"],
      "env": {
        "AZURE_TENANT_ID": "${env:AZURE_TENANT_ID}",
        "AZURE_CLIENT_ID": "${env:AZURE_CLIENT_ID}",
        "AZURE_CLIENT_SECRET": "${env:AZURE_CLIENT_SECRET}",
        "AZURE_SUBSCRIPTION_ID": "${env:AZURE_SUBSCRIPTION_ID}",
        "AZURE_RESOURCE_GROUP": "${env:AZURE_RESOURCE_GROUP}",
        "AZURE_DATA_FACTORY_NAME": "${env:AZURE_DATA_FACTORY_NAME}",
        "USE_MOCK_DATA": "false"
      }
    }
  }
}

Cursor / Windsurf

Both support MCP via ~/.cursor/mcp.json or ~/.windsurf/mcp.json using the same format as the Claude Desktop config above.

MCP Inspector (for testing and demos)

npx @modelcontextprotocol/inspector python server.py

Opens a browser UI to call each tool individually and inspect raw JSON responses. Useful for testing your setup or running demos.

Testing with MCP Inspector

MCP Inspector lets you call each tool directly in a browser UI and inspect the raw JSON responses, with no AI client needed. Great for testing your setup, debugging, or demos.

Launch (Windows)

scripts\inspect.cmd

This loads credentials from your .env file and opens the inspector automatically.

Launch (Mac/Linux)

source .env && npx @modelcontextprotocol/inspector python server.py

What you will see

1. Browser opens at http://localhost:6274

2. Green Connected indicator - server is live

3. Tools tab lists all 5 tools with their descriptions

4. Click any tool, enter arguments, hit Run Tool, and see the exact JSON your AI client receives

Example - test the cost ranking tool

Click get_top_costly_pipelines, enter:

{ "days": 30, "top_n": 8 }

Hit Run Tool. You will see your real ADF pipelines ranked by estimated monthly cost with waste percentages and trend data.

Running Tests

# All tests - no Azure credentials required (uses mock data)
USE_MOCK_DATA=true python -m pytest tests/ -v

# Specific modules
python -m pytest tests/test_cost_calculator.py -v
python -m pytest tests/test_pricing_client.py -v
python -m pytest tests/test_recommendations.py -v
python -m pytest tests/test_tools.py -v

How Costs Are Calculated

Costs are estimated from ADF activity run metadata using the same billing model as Azure:

When available, actual billed DIU-hours are read directly from billingReference in ADF activity output, not estimated from duration. Live pricing is fetched from the Azure Retail Prices API at startup and cached for 24 hours.

Optimization Rules

The recommendations engine applies 7 rules across your pipeline history:

How This Differs from azure-mcp and Azure Advisor

azure-mcp gives your AI assistant a hammer. This gives it a scalpel.

Cost Disclaimer

Cost estimates are based on Azure PAYG pricing. Actual costs vary by Azure agreement type (Enterprise Agreement, CSP, PAYG), reserved capacity, region, and currency. Always cross-check significant decisions with Azure Cost Management for exact billing figures.

V2 Roadmap

• Remediation with code: Generate ADF JSON pipeline/trigger configs as fixes, not just advice

• Multi-factory support: Query across multiple ADF instances in one conversation

• Trend analysis: Week-over-week cost trending with anomaly detection

• Databricks integration: Include Databricks job cost as part of ADF external activity cost

• Budget alerts: Set thresholds and get proactive warnings

• Remote MCP (enterprise): Deploy as remote MCP server on Azure Container Apps with Managed Identity, no local credentials, full audit trail

Contributing

Contributions welcome. Key areas:

• Additional recommendation rules (open an issue to propose)

• New Azure regions in the pricing parser

• Synapse Analytics support

• Integration tests with real ADF instances

# Dev setup
pip install -r requirements.txt
USE_MOCK_DATA=true python -m pytest tests/ -v

License: MIT