COPY FROM
A
COPY FROM job allows you to copy your data from a given source into a table created in a metastore connection. This table then serves as your staging table and can be used with SQLake transformation jobs to write to various target locations.Note that when writing to a staging table with a
COPY FROM job, the table should not have any primary keys and, if partitioned, should only be partitioned by time.Additionally, multiple
COPY FROM jobs can write to the same table which results in a UNION ALL of the data copied into the table. This means that any duplicate records that arrive are not removed from the table.Valid identifiers match the following format:
identifier = "([^"]|"")*"|[A-Za-z_][A-Za-z0-9_]*;
Table identifiers can be provided in the following format:
<catalog_name>.<schema_name>.<table_name>
Note that only tables created in metastore connections can be used in
COPY FROM jobs.Last modified 2mo ago