COPY FROMjob allows you to copy your data from a given source into a table created in a metastore connection. This table then serves as your staging table and can be used with SQLake transformation jobs to write to various target locations.
Note that when writing to a staging table with a
COPY FROMjob, the table should not have any primary keys and, if partitioned, should only be partitioned by time.
COPY FROMjobs can write to the same table which results in a
UNION ALLof the data copied into the table. This means that any duplicate records that arrive are not removed from the table.
Valid identifiers match the following format:
identifier = "([^"]|"")*"|[A-Za-z_][A-Za-z0-9_]*;
Table identifiers can be provided in the following format:
Note that only tables created in metastore connections can be used in