ds_complement_data
Backfills missing or historical workflow data for a single partition or date range, with optional task-scoped and serial/parallel execution controls.
Instructions
Backfill (complement) workflow data for date range or single partition.
Recommended usage (v2.0.14):
Single partition: partition_date="2024-01-01" (clearer semantics, recommended)
Date range: start_date + end_date (multi-partition backfill)
From specific task: start_task_names + task_depend_type="TASK_POST"
Single task only: start_task_names + task_depend_type="TASK_ONLY"
⚠️ Important (v2.0.19):
Default mode: RUN_MODE_SERIAL (one partition at a time, safer)
Parallel mode: run_mode="RUN_MODE_PARALLEL" (multiple partitions concurrently, higher risk)
Before backfill: check dependency chain with ds_get_workflow → analyze upstream deps
Minimize scope: prefer start_task_names + TASK_POST to backfill from target task forward
Mandatory standard: don't backfill entire workflow unless full-chain rerun is explicitly needed
Args: project_name: Project name workflow_code: Process-definition code start_date: Start date yyyy-MM-dd (for range backfill) end_date: End date yyyy-MM-dd (for range backfill) partition_date: Single partition yyyy-MM-dd (alternative to start_date+end_date, recommended for clarity) start_task_names: Optional list of task names to start from (backfills these + their downstream tasks) task_depend_type: Dependency type (default TASK_POST: from start tasks forward; TASK_ONLY: start tasks only; TASK_PRE: start tasks + upstream) run_mode: RUN_MODE_SERIAL (default, one at a time) or RUN_MODE_PARALLEL (concurrent)
Serial ordering guarantee (v2.0.18): In RUN_MODE_SERIAL, the request is submitted using DS's continuous range fields complementStartDate / complementEndDate, so DS generates instances strictly in ascending day order. Discrete date-list formats (complementScheduleDateList / comma-separated) do not guarantee ordering and are only used as fallbacks. The chosen format is reported in the return value's "format" field ("date_range" / "comma_separated" / "json_list").
Examples: # Single partition (recommended) ds_complement_data(workflow_code=123, partition_date="2024-01-01")
# Date range (serial mode, safer)
ds_complement_data(workflow_code=123, start_date="2024-01-01", end_date="2024-01-31")
# From specific task forward (minimize scope)
ds_complement_data(workflow_code=123, partition_date="2024-01-01",
start_task_names=["ads_table"], task_depend_type="TASK_POST")
# Parallel mode (higher concurrency, more resource usage)
ds_complement_data(workflow_code=123, start_date="2024-01-01", end_date="2024-01-10",
run_mode="RUN_MODE_PARALLEL")
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"project_name": "<project_name>",
"workflow_code": "<workflow_code>"
},
"name": "ds_complement_data"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| end_date | No | ||
| run_mode | No | RUN_MODE_SERIAL | |
| start_date | No | ||
| project_name | Yes | ||
| workflow_code | Yes | ||
| partition_date | No | ||
| start_task_names | No | ||
| task_depend_type | No | TASK_POST |