A COPY FROM job allows you to copy your data from a given source into a table created in a metastore connection. This table then serves as your staging table and can be used with SQLake transformation jobs to write to various target locations.
Note that when writing to a staging table with a COPY FROM job, the table should not have any primary keys and, if partitioned, should only be partitioned by time.
Additionally, multiple COPY FROM jobs can write to the same table which results in a UNION ALL of the data copied into the table. This means that any duplicate records that arrive are not removed from the table.


Jump to

Job identifier

Valid identifiers match the following format:
identifier = "([^"]|"")*"|[A-Za-z_][A-Za-z0-9_]*;

Job and source options

Table identifier

Table identifiers can be provided in the following format:
Note that only tables created in metastore connections can be used in COPY FROM jobs.