Release Notes 5.2.2

The 5.2.2 release provides upgrades to several existing features across Incorta. The main update in this release is to the Microsoft Azure Data Lake Source (ADLS). Additional updates included improved loading of parquet segments and node configurations for Notebook.

Release Highlights

Note

This release uses Data Agent version 2.3.0. Please make sure to upgrade to this version of the data agent.

New Features

Architecture and Application Layer

Upgraded Microsoft Azure Data Lake Source Gen2

In this release Incorta is upgrading the Microsoft Azure Data Lake Source (ADLS) SDK jar files to version 12.11.0. This upgrade aims to introduce several improvements, such as enhancing the capabilities of ADLS as a tenant storage repository, improving the performance of writing large files, and lessening memory usage.

Parallel column reading

In this release, Incorta can read parquet segments and load them into the engine memory in parallel to speed up column reading, especially string columns.

This feature enhances the performance of the following:

  • Creating the PK-index during a load job
  • Loading data into the Loader memory and Analytics memory
  • Reading columns when creating joins and formula columns
  • Creating insights and rendering dashboards
  • Warming up the Loader and Analytics services
Note

The 5.2.2 release introduces a beta version of this feature disabled by default. To enable it, edit the engine.properties file in both the Analytics Service and the Loader Service nodes and set the store.compressed_parallel and parquet.ignore_duplicates properties to true.

New internal node configurations for Notebook

Two new internal node configurations for the Incorta host and port have been added to use with Notebook. The new Notebook specific configurations ensure that Notebook is able to maintain connection with Incorta. The new configurations can be set in the analytics node by modifying the node.properties file.

The new configurations are as follows:

  • notebook.incorta.host set to POD_IP env.variable
  • notebook.incorta.port set to 8080

Additional Enhancements and Fixes

In addition to the new features and major enhancements mentioned above, this release introduces some additional enhancements and fixes that help to make Incorta more stable, engaging, and reliable.

Dashboards and Visualizations

  • Fixed an issue in which stacked column bar visualizations are too thin
  • Fixed an issue where a filter would get sorted while the user was actively renaming it. Filters will now be renamed after the user fully enters the name.

Physical Schema

  • Fixed an issue where, after upgrading to at least 5.2, the query plan would incorrectly color code table types
  • Resolved an issue where the sync of a schema model update could stall out for an extended period of time
  • Fixed an issue where filtering joins with the “Last Version” operator would cause the filter validation to never complete

Inspector Tool

  • Fixed an issue in which the inspector tool would produce an error when validating formula columns

Miscellaneous

  • Resolved an issue with the “Sync Users using LDAP” feature in which an encoding issue occurred with arabic names
  • Fixed an issue that caused the in operator not to be available when creating a regular runtime security filter
  • Resolved an issue where the add_months function could fail in execution
  • Fixed an issue where the “last used columns” in the warm-up loader of the CMC would not load tables
  • Resolved multiple Japanese translation issues in the Content Manager and the Scheduler
  • Fixed an issue where business schema views would fallback to spark when they should be served from the engine