July 2023
Upsolver new features, enhancements, and bug fixes for July 2023.
Last updated
Upsolver new features, enhancements, and bug fixes for July 2023.
Last updated
The following software updates were released in July:
Snowflake Jobs:
Support setting COMMIT_INTERVAL. This allows configuring different intervals for processing the job and for writing to Snowflake.
CDC Jobs:
(Beta) Support replication data from using change data capture.
Elasticsearch Jobs:
Support setting routing (_routing) by using the new property .
Added a new option to the property: NONE. This allows us to write to a single index name.
CDC Jobs:
Add the ability to snapshot multiple tables at once:
Snowflake Jobs:
Fixed an issue with a custom insert/update expression causing the job to fail if the field is also mapped in the select statement.
On auto-managed tables, Upsolver will not create an extra column if the following conversion happens:
Original column is double and has a value of type long.
Original column is a timestamp and has a value of type date.
Original column is a varchar.
Original column is Variant In all other cases, we will create an extra column with the new type as the column name suffix. For example: if a column col was of type bigint and got a Double value, we will create a column COL_DOUBLE in the Snowflake table.
Fixed a delay in Materialized View on Job List/Index page.
Fixed tree on fields containing dots, e.g. turning {"a\.b": 1} to {"a.b": 1}.
Snowflake Jobs:
Changed the file format to copy from Avro to JSON. This fixed an issue when ingesting records with sub-fields that have special characters.
Ingestion wizard:
Support was added for creating a PostgreSQL heartbeat table within the wizard.
Reduce the frequency of metadata queries to Snowflake in order to reduce the cost of COMPUTE SERVICES charged by Snowflake.
New UUID() function returns a unique identifier (UUID) string.
Sign-out is now available from the main screen.
PostgreSQL CDC: ignore rows from the heartbeat table.
Upgraded Debezium version from 2.1.3 to 2.2.1.
Ingestion wizard:
Supports compute cluster input (in case the organization has more than one compute cluster).
Supports basic expectation.
The cluster version appears in the UI on the Clusters page.
Snowflake table: show variant column statistics on field level.
Fixed the conversion of float to double to preserve the perceived semantic value in CDC sources and in data sources that get Avro or Parquet.
Added () - this function receives an element of an array of records and returns the 1-based index of the element position (incrementing regardless of whether the array is nested). Null values are not counted.
Added () - this function receives an element of an array of records and returns the 1-based index of the element position (index resets to 1 for each sub-array). Null values are not counted.
[BREAKING CHANGE] Fixed on fields containing dots, e.g. turning {"a.b": 1} to {"a\.b": 1}.
ROUTING_FIELD_NAMEVALUE_INDEX_IN_ROWVALUE_INDEX_IN_ARRAYRECORD_TO_JSON