Run and monitor pipelines
manage_pipeline_runStart, stop, and monitor Spark Declarative Pipeline updates, then inspect errors and event logs to troubleshoot failed runs.
Instructions
Run and monitor Spark Declarative Pipeline updates and surface pipeline errors.
start (pipeline_id[, full_refresh, refresh_selection, full_refresh_selection, validate_only, parameters, wait, timeout_seconds]): EXECUTION; returns update_id with status 'pending'. Full refreshes are also DESTRUCTIVE (confirm required) because table state is reset.
stop (pipeline_id): stops the active update (DESTRUCTIVE, confirm required).
get_update / wait (pipeline_id[, update_id] - default latest): state; failed updates include ERROR events.
list_updates (pipeline_id): update history, newest first.
list_events (pipeline_id[, level, update_id, filter]): event log, newest first.
Safety classification: depends on input (DESTRUCTIVE, EXECUTION, READ_ONLY).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| wait | No | start: poll until the update finishes (bounded). | |
| level | No | list_events: only events of this level. | |
| action | Yes | start: start an update; stop: stop the active update; get_update: one update's state (+errors if failed); list_updates: update history; list_events: event log (use level='ERROR' for errors); wait: poll an update until it finishes (bounded). | |
| filter | No | list_events: raw filter, e.g. "timestamp > '2025-01-01T00:00:00Z'". | |
| confirm | No | Set to true ONLY after the user has reviewed the plan returned by a previous call with status 'confirmation_required'. Required for destructive/security-sensitive actions. | |
| dry_run | No | If true, validate and return the planned change without executing it. | |
| page_size | No | Max items to return (server caps this). | |
| update_id | No | get_update/wait (defaults to the latest update); list_events: filter. | |
| page_token | No | next_page_token from a previous response. | |
| parameters | No | start: key/value pipeline parameters. | |
| pipeline_id | Yes | Pipeline id. | |
| full_refresh | No | start: reset ALL tables before running (DESTRUCTIVE). | |
| validate_only | No | start: only validate the source code; materialize nothing. | |
| timeout_seconds | No | Max seconds to wait (capped by server). | |
| refresh_selection | No | start: tables to refresh (incremental). | |
| full_refresh_selection | No | start: tables to fully refresh (DESTRUCTIVE). |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| data | No | ||
| page | No | ||
| plan | No | ||
| tool | Yes | ||
| action | No | ||
| safety | No | ||
| status | No | success | |
| summary | Yes | ||
| warnings | No | ||
| next_steps | No | Suggested follow-up calls. | |
| request_id | No |