UP20010 Source Data Not Found

A new ingestion job could not find data at the specified location.

Possible Causes

For example, in the bucket stores-data, no files were found in the orders folder:

CREATE SYNC JOB load_orders_raw_data_from_s3
    CONTENT_TYPE = JSON
AS COPY FROM S3 s3_connection 
    LOCATION = 's3://store-data/orders/' 
INTO default_glue_catalog.stores_data.orders_raw_data; 

Possible Solutions

• Verify that the data exists at the location specified in the job.

• Verify you configured the job to read from the correct source location (topic/stream/bucket and path/etc.).

• If data is expected to arrive in the source at a later point in time, either the SKIP_ALL_VALIDATIONS or SKIP_VALIDATIONS job option should be included in the job creation declaration. This option instructs Upsolver to ignore specific validations to enable you to create a job that reads from a source that currently has no data. The validation parameter depends on the source:

  • Amazon S3 - the source has not been created yet: SKIP_VALIDATIONS = ('EMPTY_PATH')

  • Apache Kafka - the topic has not been created yet: SKIP_VALIDATIONS = ('MISSING_TOPIC')

  • Amazon Kinesis - the stream has not been created yet: SKIP_VALIDATIONS = ('MISSING_STREAM')

  • Any other source: SKIP_VALIDATIONS = ('MISSING_DATA')

Visit the Ingestion jobs reference to learn more about the SKIP_ALL_VALIDATIONS and SKIP_VALIDATIONS job options.