Install the Iceberg Table Analyzer
This quickstart describes how to use the Upsolver CLI to diagnose performance issues within your Apache Iceberg tables.
Overview
The Iceberg Table Analysis CLI Tool evaluates your Apache Iceberg tables to identify how Upsolver optimizations can enhance efficiency. It presents a side-by-side comparison of current metrics against potential improvements in scan duration, file counts, and file sizes, providing a straightforward assessment of optimization opportunities.
The following example shows the output from running the CLI against an Iceberg table to check the current performance metrics:
Installation
iceberg-diag
can be installed using either Brew or PIP, as detailed below:
Using Brew
Execute the following commands to install iceberg-diag
via Brew:
Using PIP
Prerequisites
Python 3.8 or higher: Verify Python's installation:
Rust: check if installed:
If Rust is not installed, install it using:
To install iceberg-diag
using PIP, ensure you have the latest version of pip
:
Then, install the package with pip
Usage Instructions
Command-Line Options
-h
,--help
: Display the help message and exit.--profile PROFILE
: Set the AWS credentials profile for the session, defaults to the environment's current settings.--region REGION
: Set the AWS region for operations, defaults to the specified profile's default region.--database DATABASE
: Set the database name, will list all available iceberg tables if no--table-name
provided.--table-name TABLE_NAME
: Enter the table name or a glob pattern (e.g.,'*'
,'tbl_*'
).--remote
: Enable remote diagnostics by sending data to the Upsolver API for processing. The 'Remote' option yields more detailed analytics compared to running the process locally.
Usage
Displaying help information:
Listing all available databases in profile:
Listing all available iceberg tables in a given database:
Running diagnostics on a specific table in a specific AWS profile and region (completely locally):
Running diagnostics using
remote
option
Source Code
The source code of Iceberg diagnostic tool can be found here: https://github.com/Upsolver/iceberg-diag
Last updated