June 2023

Upsolver new features, enhancements, and bug fixes for June 2023.

Weekly Software Updates

The following software updates were released in June:

2023.06.26-03.44

✨ New Features

• User information and organization name are now displayed on the main pages with the ability to switch between organizations

⬆️ Enhancements

• [BREAKING CHANGE] Changed $row_number system field from 0-based to 1-based

• Added $row_number system column to transformation jobs

• Added $item_index system column, representing the source batch's row index. For example, in S3 sources, it will be the row index in a file

• Amazon S3 outputs now support the inclusion of the shard number in the target path. This allows the use of output shards without overwriting the output files

🔧 Bug Fixes

• Fixed a bug in the IS_DUPLICATE function that caused the wrong results when the job is running with an interval higher than 1 minute
• Fixed a bug reading Avro and Parquet files that caused fields of type Date to be ignored

---

2023.06.19-10.37

✨ New Features

• Snowflake table statistics are now available

🔧 Bug Fixes

• Fixed an issue reading from empty Kafka topics that contain empty partitions

• Fixed a bug reading Avro files that use a named type more than once

---

2023.06.12-08.57

✨ New Features

• Ingestion wizard - easy ingest to Snowflake, including:

  • Support Amazon S3 as a source

  • Support CDC from PostgreSQL as a source

• Manage Data Quality - Ingest data with expectations

⬆️ Enhancements

• Job monitoring improvements

🔧 Bug Fixes

• Snowflake Merge Jobs: enforce the ON clause expression to prevent creating an array

---

2023.06.05-11.39

⬆️ Enhancements

• PostgreSQL jobs:

  • With partitioned tables, expose data.full_partition_table_name field specifying the name of the event's original partition

• Job status page improvements