Amazon S3

This quickstart describes how to create an Amazon S3 connection.

Create a connection to Amazon S3

Before you begin transforming your data, you need to ingest it from Amazon S3 into Upsolver. Before you can ingest your data, you will need a connection.

Your connection is persistent, so you won't need to re-create it for every job. The connection is also shared with other users in your organization.

Here’s the code to create a connection to Amazon S3:

// Syntax
CREATE S3 CONNECTION <connection_identifier>
       AWS_ROLE = <AWS ROLE NAME>
       EXTERNAL_ID = <EXTERNAL ID FROM TRUST POLICY>;
       
// Example
CREATE S3 CONNECTION my_s3_connection
       AWS_ROLE = 'arn:aws:iam::111111111111:role/<upsolver-role-*'
       EXTERNAL_ID = '12345678'
       READ_ONLY = TRUE;

To break down exactly what’s happening, let’s go through the code. In the following line, you create an S3 connection named my_s3_connection:

CREATE S3 CONNECTION my_s3_connection

Next, you identify your role within your organization's framework on AWS. Please refer to this page for more information on AWS roles.

AWS_ROLE = 'arn:aws:iam::111111111111:role/<upsolver-role-*'

For future reference, your AWS_ROLE can be copied from your AWS IAM user page. After you complete this step, you should see the my_s3_connection connection in your navigation tree.

Optionally, you can include filters to display specific buckets or folders. If you don't include a filter, all data is displayed:

PATH_DISPLAY_FILTERS = ('s3://bucketname/', 's3://bucket-name/folder-path/');

---

Learn More

Please see the SQL command reference for Amazon S3 for the full list of connection options, and examples.