Monitoring

This page describes the job Monitoring tab for database sources.

The Monitoring tab provides insight into the status of your CDC job, from job level, through to individual tables within the replication. The metadata updates in real time, ensuring you have the latest information possible about the status of your job.

Job header

In the top left-hand corner of the tab is the name of your job, along with the status, e.g. Running, and color-coded icon to alert you to any errors or warnings so you can take appropriate action. If your job has an error, the error message is displayed across the top of the tab

In the top right corner, the tab displays the source and target where the job ingests the data from and to. The source includes the frequency of the extract, e.g. Extract every 1 minute, and the target includes the frequency of the load, or how often data is written, e.g. Load every 6 hours.

Click on the three-dots menu to show the options to Pause Load to Target, or Drop Job. Each option opens a modal dialog asking you to confirm your selection.

Tracking the replication status

When the CDC job begins, it first takes a snapshot (full historical load) of each table before loading changes incrementally. The number of concurrent snapshots is limited to avoid overloading the source database. As a result, each table can be in one of the following statuses:

• Pending Snapshot: Based on the SNAPSHOT_PARALLELISM setting for the job, the maximum number of tables will be snapshotting in parallel, while other tables will be pending.

• Snapshotting: A full historical load of the table from source to target is currently in progress.

• Syncing: The full historical load of the table was successfully completed, and changes are now being loaded incrementally.

Tables in the Pending Snapshot or Snapshotting status can be tracked in the Snapshots tab.

Tables that are running incrementally can be found in the Syncing Tables tab.

Snapshots

Tables with a status of Pending Snapshot or Snapshotting can be tracked in the Snapshots tab.

Track the status of tables snapshot in the Snapshots tab

For each table, you can track the following real-time KPIs:

Column	Description
Target Schema	The target schema where the data is loaded.
Target Table	The name of the target table in the destination.
Status	The current status of the snapshot. Possible values: Pending or Snapshotting. The number of snapshots running in parallel is determined by the SNAPSHOT_PARALLELISM setting in the replication job. You can adjust this setting while the job is running to allow more snapshots to run in parallel.
No. of Records	The number of records in the table that needs to be replicated. When the snapshot begins and before all data is read from the source, this number represents an estimate based on the source database statistics. If these statistics are outdated, the estimate may be inaccurate. While the number remains an estimate and not the final count, a blue 'Est.' label will be displayed next to it
%Read from source	The percentage of records read from the binlog out of the total. The actual number read is shown in brackets. Note: If the total number to read is estimated, this percentage may exceed 100% if the actual number is higher.
%Written to target	The percentage of records that have been successfully written and committed to the target out of the total. The actual number written is shown in brackets. Note: If the total number to read is estimated (indicated by a tilde), this percentage may exceed 100% if the actual number is higher.
Error	Displays the error message if there's currently an issue loading the table. Click the error message to open a modal with detailed error information.

Syncing Tables

Tables that are running incrementally can be found in the Syncing Tables tab.

At the top of the page, you can view the timestamp of the last event read from the binlog. Reading from the binlog occurs every minute. If your database consistently has changes and this timestamp is too old, it may indicate a problem.

Track the status of tables running incrementally in the Syncing Tables tab

For each table, you can track the following real-time KPIs:

Column	Description
Target Schema	The target schema where the data is loaded.
Target Table	The name of the target table in the destination.
Last Event Read	The timestamp of the last event (change) read from the binlog for the table. Reading from the binlog is performed every minute.
Pending Events	The number of events (changes) extracted from the binlog that have not been loaded to the target yet.
Write Status	While reading from the source (extract) occurs every minute, writing to the target is performed based on the interval configured when creating the job. The write status can be one of the following: Succeeded - The previous interval ended successfully, and the execution is scheduled for the next run. Loading - Data is currently being loaded to the target. Failed (Retrying) - The load is experiencing errors that prevent it from progressing. Due to the built-in retry mechanism, Upsolver automatically attempts to retry the execution from the exact point where the error occurred. Paused - Loading to the target is currently paused. Pending Extract - Writing data to the target is pending the completion of reading data from the source.
Start Time	If the job status is Succeeded, then the Start Time represents the next run time of the job. Otherwise, it indicates the time the task started running.
Error	Displays the error message if there's currently an issue loading the table. Click the error message to open a modal with detailed error information.