July 2023

Upsolver new features, enhancements, and bug fixes for July 2023.

Weekly Software Updates

The following software updates were released in July:

2023.07.31-02.02

  • Snowflake Jobs:

    • Support setting COMMIT_INTERVAL. This allows configuring different intervals for processing the job and for writing to Snowflake.

  • CDC Jobs:


2023.07.27-01.59

  • Elasticsearch Jobs:

  • CDC Jobs:

    • Add the ability to snapshot multiple tables at once:

  • Snowflake Jobs:

    • Fixed an issue with a custom insert/update expression causing the job to fail if the field is also mapped in the select statement.

    • On auto-managed tables, Upsolver will not create an extra column if the following conversion happens:

      • Original column is double and has a value of type long.

      • Original column is a timestamp and has a value of type date.

      • Original column is a varchar.

      • Original column is Variant In all other cases, we will create an extra column with the new type as the column name suffix. For example: if a column col was of type bigint and got a Double value, we will create a column COL_DOUBLE in the Snowflake table.

    • Fixed a delay in Materialized View on Job List/Index page.


2023.07.19-02.34

  • Fixed tree on fields containing dots, e.g. turning {"a\.b": 1} to {"a.b": 1}.

  • Snowflake Jobs:

    • Changed the file format to copy from Avro to JSON. This fixed an issue when ingesting records with sub-fields that have special characters.


2023.07.13-02.20

  • Ingestion wizard:

    • Support was added for creating a PostgreSQL heartbeat table within the wizard.

  • Reduce the frequency of metadata queries to Snowflake in order to reduce the cost of COMPUTE SERVICES charged by Snowflake.


2023.07.04-14.42

  • New UUID() function returns a unique identifier (UUID) string.

  • Sign-out is now available from the main screen.

  • PostgreSQL CDC: ignore rows from the heartbeat table.

  • Upgraded Debezium version from 2.1.3 to 2.2.1.

  • Ingestion wizard:

    • Supports compute cluster input (in case the organization has more than one compute cluster).

    • Supports basic expectation.

  • The cluster version appears in the UI on the Clusters page.

  • Snowflake table: show variant column statistics on field level.

  • Fixed the conversion of float to double to preserve the perceived semantic value in CDC sources and in data sources that get Avro or Parquet.

Last updated