April 2024
Upsolver new features, enhancements, and bug fixes for April 2024.
2024.04.15-07.06
⬆️ Enhancements
Introduced the
PARSE_DEBEZIUM_JSON_TYPE
property to the Avro Schema Registry content format for dynamic parsing of JSON columns from Debezium sources into Upsolver records or keeping as JSON strings. For Snowflake outputs with schema evolution, fields are written to columns of type Variant.Upgraded the Snowflake driver to 3.15.0
UI: ClickHouse wizard cosmetic changes
🔧 Bug Fixes
Fixed a bug preventing the pausing of ingestion jobs to Snowflake
Iceberg schema evolution:
Nested fields were added without the field docs, which are later used to understand which field evolved from which. Affected tables may need to be recreated if jobs writing to them are causing errors
Was not handling cases where a field can have multiple types (e.g., a field can be a record and can also be an array of strings)
2024.04.04-09.33
✨ New Features
Ingestion wizard:
ClickHouse is now supported as a target (CDC sources are not supported at this point)
⬆️ Enhancements
For new entities, you can now use the updated Parquet list structure (
parquet.avro.write-old-list-structure = false
) when writing Parquet files to S3 and Upsolver tablesSupport casting strings to JSON in jobs writing to Iceberg tables
Previewing Classic Data Sources is now supported (
SELECT * FROM "classic data source name"
)Cost reduction:
Reduced S3 API costs of Iceberg tables
Reduced S3 API costs of Hive tables
🔧 Bug Fixes
Fixed a bug that could skip data when reading from CDC sources
CDC Event log is now deleted right after parsing the log events
Increased performance of VPC integration experience
Last updated