Output to Amazon Redshift

Prerequisites

You also need a storage connection that has access to the bucket you would like the job to use to store the intermediate files used while running the job.

Finally, you should also have a staging table created previously that contains the data you intend to write to Redshift.

Create a job writing to Redshift

After you have fulfilled the prerequisites, you can create an INSERT job as follows:

CREATE JOB load_data_to_redshift
    START_FROM = BEGINNING
    SKIP_FAILED_FILES = TRUE
    FAIL_ON_WRITE_ERROR = FALSE
AS INSERT INTO REDSHIFT <redshift_connection>.<schema_name>.<target_table_name> 
    MAP_COLUMNS_BY_NAME            
    SELECT orderid AS app_name
    FROM <glue_catalog_name>.<database_name>.<table_name>
    WHERE time_filter();

This example only uses a subset of all job options available when writing to Redshift. Depending on your use case, you may want to configure a different set of options.


Learn More

Last updated