August 2023

Upsolver new features, enhancements, and bug fixes for August 2023.

Weekly Software Updates

The following software updates were released in August:


⬆️ Enhancements

  • Snowflake: SELECT * will preserve the original case of field names in variant columns.

  • SQL: Allow altering EXPOSE_IN_CATALOG property in tables.

  • Performance Improvement: Reduce the number of file operations when coordinating future table operations.

  • Write Timestamp and Date types as ISO-8601 strings in jobs that write to Elasticsearch.

  • Ingestion wizard: Support CDC from MongoDB source (Preview).

🔧 Bug Fixes

  • Jobs: When using MAP_COLUMNS_BY_NAME, the EXCEPT columns list was fixed to be case-insensitive


⬆️ Enhancements

  • Write Timestamp and Date types as ISO-8601 strings in string output jobs, for example: Job to Elasticsearch, Job to Amazon S3 with format JSON/CSV.

  • Write Timestamp and Date types as ISO-8601 in RECORD_TO_JSON function.

🔧 Bug Fixes

  • Performance improvements in CDC jobs.

  • Performance improvements when querying the Upsolver Query Engine.


⬆️ Enhancements

  • Users can now omit the connection type when specifying a source or target in jobs (e.g. INSERT INTO S3 catalog LOCATION = '...' can be replaced in INSERT INTO catalog LOCATION = '...').

  • Improved the performance of CDC jobs reading from databases with a large number of tables.

  • Querying the information schema tables now returns Jobs and Materialized Views in deleting state.

  • Elasticsearch Jobs:

    • MERGE jobs now support deleting documents by using the WHEN MATCHED AND ... THEN DELETE syntax.

  • Upgraded Avro and Parquet libraries to the latest versions.

🔧 Bug Fixes

  • Major improvements when reading from a table with a large number of partitions.


New Features

  • CDC Jobs:

    • (Beta) Support replicating data from MongoDB.

⬆️ Enhancements

  • Elasticsearch Jobs:

    • Support deleting documents with MERGE jobs.