June 2024
Upsolver new features, enhancements, and bug fixes for June 2024.
Release Notes Blog
For more detailed information on these updates, check out the Upsolver July 2024 Feature Summary blog.
2024.06.30-08.33
⬆️ Enhancements
Apache Iceberg:
Added support for creating Iceberg tables with hidden partitions using partition transforms
Added support for setting retention on partitions with type Long/Integer, enabling retention on hidden partitions that use time truncate transformations such as year/month/day/hour
Amazon S3: Reduced the number of upload file part requests by using larger parts
🔧 Bug Fixes
[BREAKING CHANGE]: Removed
$commit_time
column from all system tablesAmazon S3: Fixed cases where incomplete multipart upload files were not cleaned up
Apache Iceberg: Schema evolution was assumed for new fields with a suffix and type that match other fields
Minor bug fixes
2024.06.20-15.35
⬆️ Enhancements
Apache Iceberg:
Changed the default value for the
OPTIMIZE
property toFALSE
when creating external Iceberg tables
Allow the setting Iceberg properties using
iceberg.property.name
syntaxRemoved the
COLUMN_DATA_RETENTION
property when creating Iceberg tables, as this property is only supported in Upsolver Managed Hive tables
🔧 Bug Fixes
Minor bug fixes
2024.06.10-17.20
⬆️ Enhancements
Query Iceberg Tables - Customers can now query Iceberg tables using Upsolver's Query Engine for preview
Auto Sharding: Output files can now be split across shards within a partition
Improved the compaction speed for sorted Iceberg tables
🔧 Bug Fixes
Fixed a bug that caused jobs writing to Iceberg tables not to produce any rows when the job's source was a system table
Fixed Iceberg creating dangling files during failed compactions
CDC Inputs: Implemented deletion of intermediate files that store the metadata of the source database
External Iceberg Tables: Added support for data retention on external Iceberg tables
2024.06.04-11.16
⬆️ Enhancements
🔧 Bug Fixes
Non-Sync jobs reading from Apache Iceberg with an interval that is larger than 1 minute did not read all data from the source table
Jobs writing to Hive tables with an interval equal to or larger than 7 days could get stuck unless extra configuration was done by Upsolver support
Fixed a bug that prevented a job writing to a table with an end time from being marked as Completed when the end time was reached
2024.06.02-15.50
✨ New Features
Create mirrors of your Iceberg tables in Snowflake. Maintains an external Iceberg table in Snowflake that Upsolver keeps up-to-date by pointing to the latest snapshot of the table.
🔧 Bug Fixes
Minor bug fixes
Last updated