Output to an Amazon Redshift table

Prerequisites

Ensure that you have a Redshift connection with the correct permissions to write to your target table. Additionally, this target table and columns should already exist within Redshift before writing to it using SQLake.

You also need a storage connection that has access to the bucket you would like the job to use to store the intermediate files used while running the job.

Finally, you should also have a staging table created previously that contains the data you intend to write to Redshift.

For more information regarding ingesting your data into a staging table, see: Ingestion jobs

Create a job writing to Redshift

Once you have fulfilled the prerequisites, you can create an INSERT job as follows:

CREATE JOB load_data_to_redshift
    START_FROM = BEGINNING
    SKIP_FAILED_FILES = TRUE
    FAIL_ON_WRITE_ERROR = FALSE
    AS INSERT INTO REDSHIFT <redshift_connection>.<schema_name>.<target_table_name> MAP_COLUMNS_BY_NAME            
    SELECT orderid AS app_name
    FROM <glue_catalog_name>.<database_name>.<table_name>
    WHERE time_filter();

This example only uses a subset of all job options available when writing to Redshift. Depending on your use case, you may want to configure a different set of options.

For the full list of job options with syntax and detailed descriptions, see Redshift transformation job options.

For more details regarding the INSERT command syntax, see: INSERT