Set up a scheduled SFTP or S3 pull into an existing data spec
setup_scheduled_pullConfigures recurring file pulls from SFTP or S3 by reusing or creating connections, testing them, and scheduling loads into an analyzed spec at your chosen frequency.
Instructions
End-to-end setup for "pull files from this SFTP server / S3 bucket on a schedule": reuses a matching connection if one exists, otherwise creates one; tests it; then creates a trigger feeding an already-analyzed spec on the given frequency. Pass hostname for sftp, or roleArn (+ s3Bucket) for aws_s3.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| dedupe | No | Required — whether repeat pulls skip files already loaded into this spec. | |
| roleArn | No | aws_s3: the IAM role the customer will create/update. | |
| hostname | No | sftp: SFTP server hostname to pull from. | |
| preRules | No | Natural language: which files to pick up. | |
| s3Bucket | No | aws_s3: bucket to poll. Required when roleArn is given. | |
| s3Prefix | No | aws_s3 only. Optional key prefix. | |
| specName | Yes | Already-analyzed data spec to load files into. | |
| username | No | sftp only. Defaults to sftpuser. | |
| frequency | Yes | Pull schedule. | |
| postRules | No | Natural language: what to do after a file loads. | |
| workspaceId | No | Workspace to act on. Defaults to your only workspace if you have exactly one. |