Recent Compactions

The recent_compactions system table includes information about the partitions in your Apache Iceberg tables that Upsolver has recently run, or is running, a compaction operation on.

From a worksheet within Upsolver, run the following query:

SELECT * 
FROM system.monitoring.recent_compactions;

The system.monitoring.recent_compactions table includes the following columns:

ColumnTypeDescription

catalog

string

The name of the data catalog.

compacted_file_size_in_bytes

bigint

The size of the compacted files measured in bytes.

compaction_time

datetime

The time required to run the compaction operation.

data_files

bigint

The number of data files undergoing compaction.

data_files_size_in_bytes

bigint

The size of the data files undergoing compaction.

equality_deletes_files_size_in_bytes

bigint

The total size of Equality Delete Files applied during compaction.

equality_deletes_files_to_apply

bigint

The number of equality deletes files to apply while compacting the data. Equality deletes files are files that contain delete commands by primary keys. During the compaction deletes files are being handled.

error

string

The error message encountered during the compaction processes that caused the operation to fail.

id

string

The unique identifier for the partition.

partition

string

The unique name for the partition.

position_deletes_files_size_in-bytes

bigint

The size in bytes of position deletes files applied while compacting the data.

position_deletes_files_to_apply

bigint

The number of position deletes files to apply while compacting the data. Position deletes files are files that contain delete commands by data file name and index.

read_rows

bigint

The number of rows to read from the data files.

schema

schema

The name of the schema that the table belongs to.

shard

bigint

The shard number used for the compaction.

status

status

The status of the compaction process: either Completed, Running, or Failed.

table_name

status

The name of the table that the partition belongs to.

total_cpu_time_in_seconds

bigint

The CPU time required to perform the compaction operation.

total_shards

bigint

The total shards required to run the compaction operation on the partition.

written_rows

bigint

The number of rows written to the output data files during compaction.

Last updated