glue_s3
Analyze S3 table layouts using path heuristics to recommend optimizations and create Glue jobs for small-file consolidation.
Instructions
Deprecated. Prefer glue_job. Still registered this minor version.
Analyze S3 data layouts using path heuristics (no AWS API call).
Modes:
analyze
Path-heuristic layout suggestions. No AWS call; figures are not measured.
Parameters: s3_location (required), database_name (required),
table_name (required), data_format, query_patterns, data_size_gb
optimize
Generate comprehensive S3 optimization strategy.
Parameters: database_name (required), table_name (required),
s3_location (required), data_format, target_file_size_mb,
compression_type, enable_small_file_optimization, query_patterns
consolidate
Generate Glue job for small files consolidation.
Parameters: source_database (required), source_table (required),
target_database (required), target_table (required),
target_file_size_mb, consolidation_strategy
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| mode | Yes | ||
| table_name | No | ||
| data_format | No | parquet | |
| s3_location | No | ||
| data_size_gb | No | ||
| source_table | No | ||
| target_table | No | ||
| database_name | No | ||
| query_patterns | No | ||
| source_database | No | ||
| target_database | No | ||
| compression_type | No | snappy | |
| target_file_size_mb | No | ||
| consolidation_strategy | No | coalesce | |
| enable_small_file_optimization | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||