Overview Quickstarts How To Guides Reference Articles Support

July 2023

Upsolver new features, enhancements, and bug fixes for July 2023.

Weekly Software Updates

The following software updates were released in July:

2023.07.31-02.02

⬆️ Enhancements

  • Snowflake Jobs:

    • Support setting COMMIT_INTERVAL. This allows configuring different intervals for processing the job and for writing to Snowflake.

  • CDC Jobs:

2023.07.27-01.59

⬆️ Enhancements

  • Elasticsearch Jobs:

    • Support setting routing (_routing) by using the new property ROUTING_FIELD_NAME.

    • Added a new option to the INDEX_PARTITION_SIZE property: NONE. This allows us to write to a single index name.

  • CDC Jobs:

🔧 Bug Fixes

  • Snowflake Jobs:

    • Fixed an issue with a custom insert/update expression causing the job to fail if the field is also mapped in the select statement.

    • On auto-managed tables, Upsolver will not create an extra column if the following conversion happens:

      • Original column is double and has a value of type long.

      • Original column is a timestamp and has a value of type date.

      • Original column is a varchar.

      • Original column is Variant In all other cases, we will create an extra column with the new type as the column name suffix. For example: if a column col was of type bigint and got a Double value, we will create a column COL_DOUBLE in the Snowflake table.

    • Fixed a delay in Materialized View on Job List/Index page.

2023.07.19-02.34

🔧 Bug Fixes

  • Fixed tree on fields containing dots, e.g. turning {"a\.b": 1} to {"a.b": 1}.

  • Snowflake Jobs:

    • Changed the file format to copy from Avro to JSON. This fixed an issue when ingesting records with sub-fields that have special characters.

2023.07.13-02.20

⬆️ Enhancements

  • Added VALUE_INDEX_IN_ROW() - this function receives an element of an array of records and returns the 1-based index of the element position (incrementing regardless of whether the array is nested). Null values are not counted.

  • Added VALUE_INDEX_IN_ARRAY() - this function receives an element of an array of records and returns the 1-based index of the element position (index resets to 1 for each sub-array). Null values are not counted.

  • Ingestion wizard:

    • Support was added for creating a PostgreSQL heartbeat table within the wizard.

🔧 Bug Fixes

  • Reduce the frequency of metadata queries to Snowflake in order to reduce the cost of COMPUTE SERVICES charged by Snowflake.

  • [BREAKING CHANGE] Fixed RECORD_TO_JSON on fields containing dots, e.g. turning {"a.b": 1} to {"a\.b": 1}.

2023.07.04-14.42

✨ New Features

  • New UUID() function returns a unique identifier (UUID) string.

  • Sign-out is now available from the main screen.

⬆️ Enhancements

  • PostgreSQL CDC: ignore rows from the heartbeat table.

  • Upgraded Debezium version from 2.1.3 to 2.2.1.

  • Ingestion wizard:

    • Supports compute cluster input (in case the organization has more than one compute cluster).

    • Supports basic expectation.

  • The cluster version appears in the UI on the Clusters page.

  • Snowflake table: show variant column statistics on field level.

🔧 Bug Fixes

  • Fixed the conversion of float to double to preserve the perceived semantic value in CDC sources and in data sources that get Avro or Parquet.

Last updated