DBU Waste Finder
This read-only MCP server analyzes Databricks system tables to explain DBU spend and surface wasteful resources, then proposes (never applies) fixes.
Cost overview (
cost_summary,top_cost_drivers): total list-price cost, cost per product/day, and top-20 jobs/warehouses/clusters over 1–365 days, flagging unpriced SKUs (usd_complete,unpriced_skus).Waste detection (
find_waste): measures wasted DBUs/USD for jobs on all-purpose compute, missing/long auto-termination, idle or oversized clusters, idle warehouses, serverless failed-run share, and Delta tables needing OPTIMIZE/VACUUM; reportsobservation_daysand a 30-day projection (withheld <3 days, low-confidence <7).Spike debugging (
explain_cost_spike): attributes a single day's cost jump vs the prior 7-day average to specific jobs, warehouses, or clusters.One-shot review (
cost_review_report): combined markdown report (summary, top drivers, waste findings, proposed fixes) with tunable thresholds.Fix proposals (
propose_fix): returns a unified Asset Bundle YAML diff or reviewable SQL for cluster, warehouse, and Delta issues — refused for deleted clusters and never executed.Safety: everything is read-only against system tables, uses list price (not negotiated discounts), and returns errors with GRANT fixes rather than fake $0 totals.
Reads a Databricks workspace's system tables (billing, compute, lakeflow, query, storage) to explain where DBUs are spent and identify waste. Provides tools such as cost_summary, top_cost_drivers, and find_waste to surface costly jobs, clusters and SQL warehouses, missing auto-termination, idle/oversized compute, idle warehouses, failed serverless jobs, and Delta tables needing OPTIMIZE/VACUUM; propose_fix returns suggested fixes as a review-only diff for an Asset Bundle without applying them.
DBU Waste Finder
A free, read-only Databricks notebook that shows where your DBUs go and which resources look wasteful, using only Databricks system tables. One notebook, no install; the notebook sends nothing out of your workspace.
This repo contains two things:
notebook/– the free notebook (below).mcp/– an MCP server so Claude/Cursor can answer "why did our bill jump?" from your system tables and propose fixes as a diff to your Asset Bundle (read-only, never applied). Seemcp/README.md.
Landing page: https://markatdataandbeyond.github.io/dbu-waste-finder/
What it checks
Check | Source tables |
Cost by product and top 20 cost drivers (jobs, warehouses, clusters) at list price |
|
Jobs running on all-purpose compute, with the list-price difference to job compute (ratio from the cluster's own billed SKU and cloud in |
|
All-purpose clusters with auto-termination off (0/NULL) or above 60 min, incl. clusters deleted inside the window |
|
Idle or oversized clusters (avg CPU < 10 % and memory < 30 %) |
|
SQL warehouses with auto-stop off (0) or above 60 min, incl. warehouses deleted inside the window ( |
|
Idle SQL warehouses: uptime ( |
|
Serverless jobs: DBUs and cost per job, runs, failed runs; waste = failed-run share ( |
|
Delta tables needing |
|
Measured waste and 30-day projection | see How the projection works below |
If a SKU has no row in system.billing.list_prices (e.g. GENIE_FREE_USAGE), USD totals are the sum of the priced part,
flagged incomplete (usd_complete = false), and the unpriced SKUs/items are listed (unpriced_skus, unpriced() in the notebook,
unpriced_items in find_waste). DBUs are always shown.
Related MCP server: Azure FinOps Elite
Use
Import
notebook/dbu_waste_finder.pyinto your workspace (File → Import).Attach to any compute (serverless works) as a user with
USE CATALOGonsystemandUSE SCHEMA+SELECTonsystem.billing,system.compute,system.lakeflow,system.query(grant snippet inmcp/README.md).Adjust thresholds in the first cell, Run all. Checks whose schema isn't enabled are skipped.
How the projection works
measured_saving_usd/measured_waste_dbu: the waste the checks found inside the analysis window (USD = priced items only).observation_days: how much data the window actually covers — from the earliest record inside the lookback window insystem.billing.usage(usage_start_time) orsystem.compute.warehouse_events(event_time) until now, capped at the lookback. A workspace created yesterday has ~1 observed day even with a 30-day lookback.projected_30d_usd/projected_30d_dbu= measured ÷observation_days× 30, only whenobservation_days≥ 3. Below 3 days (or unknown) they are null andprojection_withheldsays why, e.g."need >=3 days, have 0.23".low_confidence: true(plus a note) whenobservation_days< 7 (3–7 days: projection shown but flagged). Exactly 7 is not flagged.This is an extrapolation of past waste at list price, not a promise of what you will get back.
Demo output (example data)
The snippet below is example / fixture data, not a live workspace. It was produced by running the MCP tools
(cost_summary, top_cost_drivers, find_waste, propose_fix) against the same mock system-table rows the
unit tests use.
cost_summary(days=15)
total_list_cost_usd: 1000.00 (JOBS 700 + ALL_PURPOSE 300; usd_complete=false, GENIE_FREE_USAGE unpriced)
daily: 2026-10-01 → $33.30
top_cost_drivers(days=15)
job etl-nightly $500.00
cluster c-shared-01 $300.00
warehouse wh-sql-01 $120.00
find_waste(days=15)
measured_saving_usd: 203.15 measured_waste_dbu: 4.5
observation_days: 15.0 projected_30d_usd: 406.30
jobs_on_all_purpose: c-shared-01 (2 jobs, est. $165 if moved to job compute)
missing_autotermination: c-adhoc-02 (auto_termination_minutes=0)
idle_or_oversized: c-idle-03 (CPU 4 %, mem 12 %, est. $35)
idle_warehouse: wh-sql-01 (98.4 % idle, 4.0 DBU → $2.80)
delta_needs_optimize: main.bronze.events (500 files, ~4 MiB avg)
delta_needs_vacuum: main.bronze.events (40 days since vacuum, ~3 GiB reclaimable)
delta_overfrequent_optimize: main.silver.orders (14 OPTIMIZE runs / 7 days)
propose_fix(issue="delta_optimize", resource_key="main.bronze.events")
-- Proposal only; review before running. DBU Waste Finder never executes this.
OPTIMIZE main.bronze.events;Live trial run (real workspace, 2026-10-29)
Not fixture data. Same tools (cost_summary, top_cost_drivers, find_waste, cost_review_report) against a Databricks trial workspace over serverless SQL (Statement API + system tables only).

Field | Value |
Observed days | 2.84 (30-day lookback requested) |
Total list-price cost | $18.17 ( |
Measured waste | $3.50 / 5.0 DBU ( |
30-day projection | withheld (needs ≥ 3 observed days) |
Classic clusters / Delta | empty on this trial — checks ran, no data |
Honesty: trial workspace, little real usage, list price not a negotiated discount. Full markdown/PDF of this run lives in the local waste-test/ folder used for TDD; the image above is the executive summary only.
Privacy
Notebook: runs inside your workspace, read-only; nothing leaves the workspace.
MCP server: the MCP server runs on your machine, not in the workspace. It reads system tables with your token and returns the results to your AI client (Claude, Cursor, …) and thereby to that AI provider: job/cluster/warehouse names and ids, DBUs, list-price amounts and owners' email addresses (
missing_autoterminationreturnsowned_by). Use a read-only service principal and only connect it to an AI client you may share that metadata with. Nothing is sent to Data and Beyond.
Caveats
Costs use
pricing.effective_list.default(list price), not your contracted discount.For idle/oversized clusters, 50 % of their cost is counted as waste (an assumption; tune
IDLE_SAVING_SHARE). Checks can overlap.Idle-warehouse DBU/h per size (2X-Small 4 … 4X-Large 528) is the published Databricks SQL rate, an assumption; real billed DBUs are in
system.billing.usage.system.billing.usagelags by hours;system.computeandsystem.queryusually land sooner, so warehouse checks can show results before cost does.usage_metadata.job_idis not populated for jobs on all-purpose compute (per Databricks docs), so those are detected viajob_task_run_timeline.compute_ids.system.billingis account-wide, butsystem.computeandsystem.lakeflowonly cover workspaces in the current region. Cluster usage billed in workspaces that this region can't see is listed undercluster_region_gapwith a warning. Run the checks once per region.
Sources (fetched 2026-10-26)
Billable usage: https://docs.databricks.com/aws/en/admin/system-tables/billing
Pricing: https://docs.databricks.com/aws/en/admin/system-tables/pricing
Compute: https://docs.databricks.com/aws/en/admin/system-tables/compute
Jobs: https://docs.databricks.com/aws/en/admin/system-tables/jobs
Test locally
From the repo root (Python ≥ 3.10):
python3 -m venv .venv && .venv/bin/pip install -r requirements-test.txt -e ./mcp && .venv/bin/pytest -qRuns the notebook test (every query parsed as Databricks SQL with sqlglot and executed on mock system tables in DuckDB), the MCP server tests (mocked query results) and landing-page checks. No workspace needed.
Validation status
Run live against a Databricks trial workspace (AWS, serverless-only) on 2026-10-26: all queries execute;
idle_warehousefound the test warehouse (77 min up, 2 busy min);warehouse_long_autostopfound it after deletion; cost tools return DBUs and a partial USD total for priced SKUs ($6.89 at ~16:40Z) withusd_complete: falseand the one unpriced SKU listed (GENIE_FREE_USAGE, 0.326 DBU).Not yet validated live: the classic-cluster checks (jobs on all-purpose, auto-termination, idle clusters) — the trial workspace cannot run classic clusters.
Idle-warehouse USD verified live ~16:40Z: billed SKU PREMIUM_SERVERLESS_SQL_COMPUTE_US_WEST_OREGON at $0.70/DBU → 5.0 idle DBU = $3.50 (billing: 3.88 DBU for 13:00–14:00Z, consistent with 4 DBU/h for 2X-Small).
serverless_jobsverified live 2026-10-26 ~16:40Z: test job 4 runs / 0 failed (=job_run_timeline), 0.215 DBU and $0.08 (=system.billing.usage0.21454 DBU × $0.35). Billing then covered 13:00–14:00Z only (runs at 13:15 and 13:45Z), so the DBUs of the 14:15/14:45Z runs are not yet confirmed.
Not affiliated with Databricks. "Databricks" is a trademark of Databricks, Inc. MIT licensed, © 2026 Data and Beyond ApS.
Available Tools
6 toolscost_review_reportA
One markdown cost-review report for the last days days (1..365) from cost_summary +
top_cost_drivers + find_waste: executive summary, top drivers, waste findings with estimated
list-price amounts, and proposed fixes. max_autotermination_minutes / warehouse_idle_pct /
max_rows are forwarded to find_waste. Read-only; proposals only — nothing is applied.
Returns {days, markdown, measured_saving_usd, projected_30d_usd}.
Inputs: days, max_rows, warehouse_idle_pct, max_autotermination_minutes.
cost_summaryA
Orient on spend: total list-price cost, cost per product and per day for the last days days
(integer 1..365). Read-only; list price, not your discount. USD totals cover priced usage only —
when a SKU has no list price, usd_complete is false and unpriced_skus lists it. Use before
find_waste when you need the overall bill shape.
Inputs: days.
explain_cost_spikeA
Explain a one-day cost spike: which jobs/warehouses/clusters drove list-price cost on date
(YYYY-MM-DD UTC billing day = system.billing.usage.usage_date) vs the prior 7-day daily average.
Read-only. Use when the user asks "why did Tuesday cost more?" after cost_summary.
Inputs: date (required).
find_wasteA
Find wasted DBUs over the last days days (1..365). Read-only; never changes the workspace.
Returns measured_saving_usd / measured_waste_dbu, observation_days, projected_30d_usd/_dbu
(= measured / observation_days * 30; withheld below 3 days) and low_confidence (< 7 days).
Classic: jobs on all-purpose, clusters with auto-termination off or above
max_autotermination_minutes (default 60), idle/oversized clusters. Serverless: warehouses with
no/long auto-stop, idle warehouses (idle_pct ≥ warehouse_idle_pct, default 80), serverless jobs
incl. failed-run share. Delta: needs OPTIMIZE / VACUUM / over-frequent OPTIMIZE (proposals only).
At most max_rows rows per check (1..1000). USD is None when unknown; failed checks go under
errors with the table and the GRANT/fix — totals are never a fake $0.
Inputs: days, max_rows, warehouse_idle_pct, max_autotermination_minutes.
propose_fixA
Propose a fix as a unified diff (Asset Bundle YAML) or reviewable SQL — never applied.
issue: 'all_purpose_job' (move tasks off existing_cluster_id onto a job cluster),
'no_autotermination' (set resources.clusters..autotermination_minutes),
'warehouse_autostop' (set resources.sql_warehouses..auto_stop_mins, default 10),
or Delta (delta_optimize|delta_vacuum|delta_zorder|delta_predictive_optimization|
delta_overfrequent_optimize) which returns SQL (resource_key = catalog.schema.table;
for delta_zorder pass columns as comma-separated node_type_id; for delta_vacuum pass
retention hours as minutes, default 168 / min 7 days). Pass cluster_id from find_waste for
cluster fixes — a deleted cluster is refused. bundle_yaml may be empty for Delta issues.
Inputs: issue (required), bundle_yaml (required), resource_key (required), path, +5 more. Returns: result
top_cost_driversA
Top 20 jobs, warehouses and clusters by list-price cost over the last days days (1..365).
Read-only; returns {rows, usd_complete, unpriced_skus}. Use to answer "what spent the most?"
before drilling into waste with find_waste. List price only — not applied discounts.
Inputs: days.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 6 tools
Each tool targets a distinct resource and question: cost_summary (bill shape), top_cost_drivers (top spenders), cost_review_report (composite deliverable), find_waste (waste detection), explain_cost_spike (single-day cause), and propose_fix (remediation output). Descriptions explicitly establish call ordering and dependency (e.g. use before find_waste, after cost_summary), so an agent can select confidently.
All names are snake_case and readable, but conventions mix noun phrases (cost_summary, top_cost_drivers, cost_review_report) with verb_noun phrases (find_waste, explain_cost_spike, propose_fix). The deviation is minor and does not impede understanding.
Six tools is well-scoped for a cost-review/waste-finder server, with each tool earning its place across orient, drill-down, report, explanation, and remediation. No redundant or trivial entries.
The surface covers orientation, drill-down, spike explanation, comprehensive reporting, and fix proposals, forming a coherent read-only workflow. Minor gaps remain, such as no direct entity-detail lookup (specific job/cluster metadata) or validation of proposed fixes, but agents can work around these.
Maintenance
Related MCP Connectors
LLM cost optimization gateway, repo token-waste auditor & model arbitrage MCP server.
- Costory FinOps MCPDeprecatedio.costory
FinOps harness MCP: allocated cloud, data and LLM bills. Build on it or switch. Cloud read-only.
Hosted MCP server for AWS cloud spend: service breakdowns, anomalies, savings and forecasts.
Read-only AI/LLM cost tools with current MCP discovery and attributable Registry transport.
Related MCP Servers
- AlicenseCqualityDmaintenanceA read-only MCP server that enables users to query Databricks SQL, browse metadata, and monitor Delta Lake tables. It also supports tracking Databricks Jobs, DLT Pipelines, and cluster metrics through natural language interfaces.254MIT
- AlicenseNot gradedqualityDmaintenanceProduction-grade MCP server for enterprise Azure cost optimization, enabling spend anomaly detection, multi-tenant auditing, budget validation, and compliance-aware recommendations.1MIT
- AlicenseAqualityBmaintenanceLocal-first FinOps MCP server. Ask about your AWS, Azure, GCP, and SaaS costs in plain English. Anomaly detection, rightsizing, idle-resource cleanup, and Jira/Linear ticketing. Credentials never leave your machine.1020Apache 2.0
- FlicenseNot gradedqualityDmaintenanceA governed MCP server exposing Databricks operations as tools for jobs orchestration, SQL execution, notebook creation, Unity Catalog governance, lineage, clusters, and DLT pipelines, with safety features like dry-run and audit logging.-