ds_modify_workflow_dag
Modify existing workflow DAGs by adding, deleting, or updating tasks and adjusting dependencies. Automatically manages offline/online state and keeps previous versions for rollback.
Instructions
Modify existing workflow DAG (add/delete/update tasks + adjust dependencies).
Implementation: read current DAG → apply operations → write back.
⚠️ Risk warning:
- Modifications create a new version; old versions preserved and rollback-able
- If workflow is running, it will be taken offline first (auto_offline=True by default)
- Auto-re-online after modification (auto_online=True by default)
- DS has no native "add single task" API; this tool simulates it via "read → modify → full update"
Args:
project_name: Project name
workflow_code: Workflow code
operations: Operation list (supports 5 action types)
auto_offline: Auto-offline before modification (default True)
auto_online: Auto-online after modification (default True)
auto_online_schedule: Auto-restore associated schedule after onlining (default True)
⚠️ DS offlines the schedule when workflow goes offline;
set False only if schedule auto-restore is not needed.
Supported operation types:
[1] Add task (add_task):
{
"action": "add_task",
"task": {
"name": "wait_upstream",
"type": "DEPENDENT",
"depend_items": [{
"project_code": 123,
"definition_code": 456,
"dep_task_code": 0, # 0=entire workflow, nonzero=specific task
"cycle": "day", # day/hour/week/month
"date_value": "today" # today/yesterday/last7days
}],
"dep_relation": "AND" # multi-dep relation: AND/OR
# other fields same as ds_create_dag_workflow
},
"connect_from": "task_a", # predecessor task name (empty=start node)
"connect_to": "task_b" # successor task name (empty=end node)
}
[2] Delete task (delete_task):
{
"action": "delete_task",
"task_name": "old_task",
"reconnect": true # auto-reconnect predecessor/successor to prevent DAG break
}
[3] Update task parameters (update_task):
{
"action": "update_task",
"task_name": "my_sql",
"updates": {
# ===== Common fields (snake_case / camelCase aliases supported) =====
"name": "new_task_name", # v2.0.11: rename node
"description": "Updated description",
"fail_retry_times": 3, # or "failRetryTimes"
"fail_retry_interval": 4, # or "failRetryInterval" (v2.0.10 fixed)
"timeout": 300, # timeout duration (minutes)
"timeout_flag": "OPEN", # or "timeoutFlag": "OPEN"/"CLOSE" or True/False
"timeout_notify_strategy": "FAILED", # or "timeoutNotifyStrategy": "FAILED"/"WARNING"
"worker_group": "default", # or "workerGroup"
"task_priority": "HIGH", # or "taskPriority": "HIGH"/"MEDIUM"/"LOW"
"delay_time": 0, # or "delayTime" (delay in minutes)
"flag": "YES", # "YES"=enabled "NO"=disabled
# ===== SQL task =====
"sql": "SELECT * FROM new_table WHERE dt='$[yyyyMMdd-1]'",
"datasource_id": 1, # or "datasource"
"sql_type": "HIVE", # "HIVE"/"MYSQL"/"POSTGRESQL" etc.
"sql_type_select": 0, # or "sqlType": 0=query 1=non-query
"pre_statements": [ # or "preStatements" (pre-SQL list)
"SET hive.exec.dynamic.partition=true",
"SET spark.sql.sources.partitionOverwriteMode=dynamic"
],
"post_statements": ["SELECT 1"], # or "postStatements" (post-SQL list)
# ===== SHELL/PYTHON task =====
"script": "#!/bin/bash
export PATH=/xxx/bin:$PATH python3 script.py", # or "rawScript"
# ===== Resource refs (usable by SHELL/PYTHON/SQL) =====
"resource_list": [67, 58], # or "resourceList" (int ID recommended)
# or ["/public/check_partition.py"] (full path, starting with /)
# ===== Local params =====
"local_params": [{"prop":"dt","value":"$[yyyyMMdd-1]"}] # or "localParams"
}
}
⚠️ resource_list format:
- Recommended: resource_id (int): [67, 58]
- Or full path (str): ["/public/check_partition.py", "/scripts/sync.py"]
- Paths must start with /, e.g. /public/xxx.py not public/xxx.py
- Use ds_list_resources() to view all resources with ids and paths
⚠️ Field naming compatibility:
- Supports both snake_case (e.g. pre_statements) and camelCase (e.g. preStatements)
- Both styles work identically; snake_case recommended
- Unrecognized fields logged in return value's ignored_fields
[4] Update node coordinates (update_location):
{
"action": "update_location",
"task_name": "my_task",
"x": 500,
"y": 300
}
[5] Adjust dependencies (update_relations):
{
"action": "update_relations",
"changes": [
{"op": "add", "from": "task_a", "to": "task_b"},
{"op": "remove", "from": "task_c", "to": "task_d"}
]
}
Returns:
{
"workflow_code": int,
"operations_applied": int,
"task_count_before": int,
"task_count_after": int,
"status": "updated",
"warning": risk notice
}
Typical scenario: append a DEPENDENT node so the workflow waits on upstream
ds_modify_workflow_dag(
project_name="my_project",
workflow_code=21505676237440,
operations=[{
"action": "add_task",
"task": {
"name": "wait_upstream_flow",
"type": "DEPENDENT",
"depend_items": [{
"project_code": 21582927260160,
"definition_code": 21505676200000,
"cycle": "day",
"date_value": "today"
}]
},
"connect_from": "", # start node
"connect_to": "existing_task_1" # connect to an existing task
}]
)
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| operations | Yes | ||
| auto_online | No | ||
| auto_offline | No | ||
| project_name | Yes | ||
| workflow_code | Yes | ||
| auto_online_schedule | No |