Amazon S3 over SQS data source
This article provides a guide on how to create an Amazon S3 over SQS data source in Upsolver.
1. From the Data Source page, click New.
2. Select S3 Over SQS.
3. Name this data source.
5. (Optional) Enter in the prefix that is used to determine whether to create a notification on the event that was published to the S3 bucket.
6. (Optional) Enter in the suffix that is used to determine whether or not to create a notification on the event that was published to the S3 bucket.
7. Select the content format. This is typically auto-detected, but you can manually select a format.
10. (Optional) Under Start Ingestion From, select the time from which to start ingesting the files. It is important to set this option so that the data is spread out properly over the date range.
11. (Optional) Under End Read At, select the time to stop reading the data. This is useful if you want to stop processing a data stream.
13. If enabled, specify a retention period in Upsolver for the data.
14. Click Continue. A preview of the data will appear.
15. For CSV, select a Header.
16. Click Continue again.
17. (Optional) If there are any errors, click Back to change the settings as required.
18. Click Create.