LogoLogo
OverviewQuickstartsHow To GuidesReferenceArticlesSupport
Overview
Overview
  • Welcome to Upsolver
  • GET STARTED
    • What is Upsolver?
    • Schedule a Demo
    • Start Your Free Trial
    • Apache Iceberg
  • RESOURCES
    • Reference
    • Iceberg Academy
    • Blog
    • Chill Data Summit
    • Community
    • Videos
  • RELEASE NOTES
    • March 2025
    • February 2025
    • January 2025
    • Earlier Releases
      • 2024
        • December 2024
        • November 2024
        • October 2024
        • September 2024
        • August 2024
        • July 2024
        • June 2024
        • May 2024
        • April 2024
        • March 2024
        • February 2024
        • January 2024
      • 2023
        • December 2023
        • November 2023
          • Deprecated Job Option
        • October 2023
        • September 2023
        • August 2023
        • July 2023
        • June 2023
        • May 2023
  • Legal
Powered by GitBook
On this page
  • 2024.06.30-08.33
  • 2024.06.20-15.35
  • 2024.06.10-17.20
  • 2024.06.04-11.16
  • 2024.06.02-15.50
  1. RELEASE NOTES
  2. Earlier Releases
  3. 2024

June 2024

Upsolver new features, enhancements, and bug fixes for June 2024.

Last updated 10 months ago

Release Notes Blog

For more detailed information on these updates, check out the blog.

2024.06.30-08.33

⬆️ Enhancements

  • Apache Iceberg:

    • Added support for creating Iceberg tables with hidden partitions using partition transforms

    • Added support for setting retention on partitions with type Long/Integer, enabling retention on hidden partitions that use time truncate transformations such as year/month/day/hour

  • Amazon S3: Reduced the number of upload file part requests by using larger parts

  • Targets: Added TRUNCATE_COLUMNS property to truncate VARCHAR or CHARACTER VARYING fields by the target column size when inserting into an existing table

🔧 Bug Fixes

  • [BREAKING CHANGE]: Removed $commit_time column from all system tables

  • Amazon S3: Fixed cases where incomplete multipart upload files were not cleaned up

  • Apache Iceberg: Schema evolution was assumed for new fields with a suffix and type that match other fields

  • Minor bug fixes

2024.06.20-15.35

⬆️ Enhancements

  • Apache Iceberg:

      • Changed the default value for the OPTIMIZE property to FALSE when creating external Iceberg tables

      • Allow the setting Iceberg properties using iceberg.property.name syntax

      • Removed the COLUMN_DATA_RETENTION property when creating Iceberg tables, as this property is only supported in Upsolver Managed Hive tables

🔧 Bug Fixes

  • Minor bug fixes

2024.06.10-17.20

⬆️ Enhancements

  • Query Iceberg Tables - Customers can now query Iceberg tables using Upsolver's Query Engine for preview

  • Auto Sharding: Output files can now be split across shards within a partition

  • Improved the compaction speed for sorted Iceberg tables

🔧 Bug Fixes

  • Fixed a bug that caused jobs writing to Iceberg tables not to produce any rows when the job's source was a system table

  • Fixed Iceberg creating dangling files during failed compactions

  • CDC Inputs: Implemented deletion of intermediate files that store the metadata of the source database

  • External Iceberg Tables: Added support for data retention on external Iceberg tables


2024.06.04-11.16

⬆️ Enhancements

🔧 Bug Fixes

  • Non-Sync jobs reading from Apache Iceberg with an interval that is larger than 1 minute did not read all data from the source table

  • Jobs writing to Hive tables with an interval equal to or larger than 7 days could get stuck unless extra configuration was done by Upsolver support

  • Fixed a bug that prevented a job writing to a table with an end time from being marked as Completed when the end time was reached


2024.06.02-15.50

    • Create mirrors of your Iceberg tables in Snowflake. Maintains an external Iceberg table in Snowflake that Upsolver keeps up-to-date by pointing to the latest snapshot of the table.

🔧 Bug Fixes

  • Minor bug fixes

statement:

statement:

Implemented deletion of temporary intermediate files in after their usage. This feature will be enabled gradually

:

When creating via the wizard, we have now added the ability to map selected source schemas to target schemas automatically. If target schemas do not already exist, they will be created automatically. The source and target schemas will be identical. This option was previously available only via SQL. In addition, we have improved the experience when configuring source to schema mapping

Fixed the shard number in

outputs now support custom commit interval

New Features

Support direct ingestion to

✨
Upsolver July 2024 Feature Summary
Iceberg Table Mirrors
Ingestion wizard
replication (CDC) jobs
Data Ingestion Wizard
Amazon Redshift
Amazon Redshift
CREATE EXTERNAL ICEBERG TABLE
CREATE ICEBERG TABLE
ingestion jobs
system.monitoring.recent_compactions
Amazon Redshift