Job Configuration for CDC

General job configuration

In the Name your job field, provide the name you want to use should you want to change the default name provided by Upsolver.

Then, set how often you want to update the target. By default, the job will check the source for new events every minute, and you can set the target write interval in Minutes, Hours, or Days.

Frequent writes provide up-to-date information but may be costly, especially for Snowflake.

The Replication Mode specifies how changes in the source should be handled in the target. Select from the following options:

• Exact replication: applies inserts, updates, and deletes on the target as per the source.

• Soft deletion: marks rows deleted in the source as deleted in a column named UPSOLVER_IS_DELETED in the target. Optionally, you can rename this column.

• Append only: appends (inserts) the record to the target, indicating the record type in a dedicated column named UPSOLVER_RECORD_TYPE (you can rename this column). The column values will be one of the following: Insert, Update, Delete, Snapshot. Note: This option is only supported for the following targets: Snowflake, Glue Data Catalog and Tabular.

Upsolver uses distributed locking technology to keep events strongly ordered. The ingestion timestamp is stored in a dedicated column reflecting the order. By default, the column name is UPSOLVER_EVENT_TIME, but you can rename it.

Configure your job that ingests data from a CDC source.