glue_job
Generate and manage AWS Glue job configurations and templates, including SQL-to-PySpark conversion and DynamicFrame code.
Instructions
Generate and manage AWS Glue job configurations and templates.
Modes:
template
Generate a complete AWS Glue job template.
Parameters: sql_query, job_name, source_database, source_table,
target_database, target_table, output_dir, source_format,
target_format, include_bookmarking, template_type, script_name
dynamic_frame
Convert PySpark DataFrame code to use DynamicFrames.
Parameters: pyspark_code (required), source_database, source_table,
target_database, target_table
properties
Generate Glue job properties for AWS CLI/SDK/Terraform.
Parameters: job_name (required), job_type, worker_type,
number_of_workers, max_retries, timeout, glue_version,
enable_continuous_logging, enable_metrics, enable_spark_ui
sql_conversion
Generate a Glue job that includes SQL-to-PySpark conversion.
Parameters: sql_query (required), job_name (required),
source_database, source_table, target_database, target_table,
source_format, target_format, include_bookmarking
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| mode | Yes | ||
| timeout | No | ||
| job_name | No | ||
| job_type | No | etl | |
| sql_query | No | ||
| output_dir | No | ||
| max_retries | No | ||
| script_name | No | ||
| worker_type | No | G.1X | |
| glue_version | No | 4.0 | |
| pyspark_code | No | ||
| source_table | No | ||
| target_table | No | ||
| source_format | No | parquet | |
| target_format | No | parquet | |
| template_type | No | standard | |
| enable_metrics | No | ||
| enable_spark_ui | No | ||
| source_database | No | ||
| target_database | No | ||
| number_of_workers | No | ||
| include_bookmarking | No | ||
| enable_continuous_logging | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||