Emit SIEM connector for an offload bucket
log10x_siem_connectorGenerate the pull-side connector config that lets Datadog or Splunk ingest log10x offload NDJSON from your S3 bucket.
Instructions
Generate the customer-applied SIEM-side connector config (Datadog Forwarder Lambda, or the Splunk Add-on for AWS SQS-based S3 input) that makes a SIEM pull log10x's offload output from the customer's own S3 bucket. Call this when the user has offloaded a pattern's data to S3 (the offload action / a log10x_offload_add destination) and asks how to get Datadog or Splunk to ingest it, e.g. 'ship my offload bucket to Splunk' or 'wire Datadog to read these logs'. log10x deliberately never pushes to a SIEM (vendor-driven re-ingest collides with Datadog Rehydration billing, Splunk HEC permissions, and Elastic _bulk limits); it lands NDJSON in the customer bucket and the SIEM pulls, and this tool emits exactly that pull-side connector. Args: siem (required: datadog / splunk / both), bucket (required: the offload bucket), account_id (required), region (default us-east-1), prefix (optional, default app/), plus per-SIEM options (datadog_forwarder_arn, sqs_queue_name, sourcetype, index). Returns, per target: the discovery model, ordered setup steps, paste-ready terraform (an SNS fan-out plus IAM, and for Splunk the SQS queue and the S3-to-SNS-to-SQS notification), the SIEM-native config (the Splunk inputs.conf aws_sqs_based_s3 stanza), and verified caveats. Recipes are matched to log10x's actual offload output (uncompressed NDJSON, one event per line, Hive-partitioned keys) and encode the key gotcha: the offload bucket usually already notifies the 10x indexer's SQS, and S3 allows only one notification config per overlapping prefix, so both recipes fan out via SNS rather than a second raw bucket notification. Honest caveats are surfaced: Datadog is logs-only (use log10x_backfill_metric for per-pattern metrics), and the Splunk SQS-based input collects only objects created after setup (backfill older data with a Generic S3 input). Read-only: it generates config and does not create AWS resources or mutate any env document.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| siem | Yes | Which SIEM should pull the offload: datadog (Forwarder Lambda), splunk (Add-on SQS-based S3 input), or both. | |
| index | No | Splunk index for the input. Default "main". | |
| bucket | Yes | The offload bucket the SIEM should read (where 10x writes the NDJSON). | |
| prefix | No | Offload key prefix the SIEM should watch. Default "app/". | |
| region | No | AWS region of the bucket. | us-east-1 |
| account_id | Yes | AWS account ID that owns the bucket (same account the SIEM connector runs in). | |
| sourcetype | No | Splunk sourcetype for the input. Default "log10x:offload". | |
| sqs_queue_name | No | Name for the SQS queue the Splunk input drains (splunk only). | |
| datadog_forwarder_arn | No | ARN of an already-deployed Datadog Forwarder Lambda (datadog only). Omit to leave a placeholder. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| data | No | ||
| tool | Yes | ||
| view | No | summary | |
| images | No | ||
| actions | No | ||
| summary | Yes | ||
| warnings | No | ||
| truncated | No | ||
| next_cursor | No | ||
| render_hint | No | ||
| generated_at | Yes | ||
| schema_epoch | Yes | ||
| schema_version | Yes |