Expire Snapshots

Snapshots in Iceberg tables represent consistent views of the table's data at a specific point in time. Each snapshot is created whenever changes are made to the table, such as adding, updating, or deleting data. These snapshots enable features like time travel, allowing users to query the table as it existed at a previous state.

Over time, as new snapshots are created, old ones may become irrelevant or redundant, consuming unnecessary storage. Snapshot expiration is the process of removing these outdated snapshots based on predefined policies, freeing up storage while maintaining the table's integrity and performance.

The Expire Snapshots tab lets you monitor the status of snapshot expiration in the table. It tracks the expiration process using the following parameters:

1. history.expire.max-snapshot-age-ms: The maximum age of snapshots to retain on the table and its branches during expiration.

2. history.expire.min-snapshots-to-keep: The minimum number of snapshots to retain on the table and its branches during expiration.

When the expire snapshots job runs, it checks for snapshots eligible for expiration based on these parameters. Once the job completes, the expired snapshots are deleted, and their storage is reclaimed.

Expire Snapshots Tab

Per Snapshot Expiration Job run, you will be able to track the following KPIs:

Metric	Description
Job Start time	The timestamp indicating when the snapshot expiration job started.
Status	The current status of the snapshot expiration job. Possible values include: "Running", "Completed", "Failed (Retrying)", "Failed".
Duration	The total run time duration of the job.
Snapshots Expired	The total number of snapshots that were expired during the job execution.
Latest Expired Snapshot	The timestamp of the most recent snapshot that was expired during the job.
Data Files Deleted	The total number of data files that were deleted as a result of expiring the snapshots.
Metadata files Deleted	The total number of metadata files that were deleted during the snapshot expiration process.
Storage Size Deleted	The total amount of storage space (in bytes) freed by deleting the expired snapshots, data files, and metadata files.
Errors	Errors text in case errors were detected.