glue_data
Generate AWS Glue data processing jobs for incremental, CDC, and bookmark modes. Configure source and target databases, transformation SQL, and more.
Instructions
Generate AWS Glue data processing jobs — incremental, CDC, bookmarks.
Modes:
incremental
Generate Glue job with incremental processing and job bookmarking.
Parameters: source_database (required), source_table (required),
target_database (required), target_table (required),
incremental_column (required), incremental_strategy,
transformation_sql
cdc
Generate Change Data Capture (CDC) Glue job.
Parameters: source_database (required), source_table (required),
target_database (required), target_table (required),
cdc_column, cdc_strategy, primary_keys
bookmarks
Generate job bookmark configuration for Glue jobs.
Parameters: job_name (required), bookmark_strategy,
transformation_context_keys
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| mode | Yes | ||
| job_name | No | ||
| cdc_column | No | last_modified | |
| cdc_strategy | No | upsert | |
| primary_keys | No | ||
| source_table | No | ||
| target_table | No | ||
| source_database | No | ||
| target_database | No | ||
| bookmark_strategy | No | enable | |
| incremental_column | No | ||
| transformation_sql | No | ||
| incremental_strategy | No | timestamp | |
| transformation_context_keys | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||