Release Notes 2022.9.x

Release Highlights

Incorta is introducing multiple enhancements in this release. You can now format a measure based on the value of another measure. Parallel column reading greatly enhances the performance of data loading and dashboard rendering. In addition, our new MV Assistant can recommend Spark settings to better enhance materialized view (MV) loading time and performance.

Enhancements include tracking the dependencies of formula columns in physical schema tables and MVs, coloring data in Tag Cloud insights according to other measures, and testing and executing a portion of the SQL script of Incorta SQL Views and Tables, custom SQL tables, and SQL-based data sources. A new data application marketplace is available to paid accounts upon subscription and to all trial users.

The 2022.9.1 Maintenance Pack includes additional issue resolutions.

Note

The 2022.9.0 release is available for newly created clusters and new trial users. For already existing customers upgrading from a prior release, please contact Incorta Support to schedule the upgrade.

Important

This release uses the Data Agent version 5.0.0. Please make sure to upgrade to the data agent version available on the Cloud Admin Portal.

New Features

Want More Details...

Refer to What's New in Incorta Cloud 2022.9.0 for more details and examples.

Conditional formatting based on another measure

For chart and table insights that support conditional formatting (Bar, Column, KPI, Listing Table, Aggregated Table, and Pivot Table insights), you can now apply different formats on a measure (column or formula) depending on the value or the aggregated value of another column or a formula. You can add multiple conditional formatting rules to the same measure and arrange them according to the order of application.

The Tag Cloud visualization now has the Color By tray to enable you to use coloring as an additional dimension to view your data.

For more information, refer to Visualizations → Tag Cloud.

New AlloyDB Connector

In this release, Incorta is introducing a new connector for preview that integrates with the Google-powered AlloyDB. AlloyDB is a PostgreSQL-based distributed database that is available for preview (Pre-GA). It combines performance, safety, availability, and scalability features.

Note

The Incorta AlloyDB connector is available for preview only.

For more information, refer to Connectors → AlloyDB.

Data Applications Marketplace

A new data applications marketplace is now available to view and download different data applications that either Incorta or Incorta partners provide. You will also be able to upload and share your own data applications as an Incorta Partner.

Note

The availability of the marketplace depends on your Incorta subscription.

Tracking formula column dependencies

You can now access the Data Lineage Viewer of formula columns in physical schema tables and materialized views (MVs). Track the dependencies of these formula columns and preview the list and diagram of their dependent entities.

Executing a portion of an SQL script

To facilitate the iterative process of writing SQL in Incorta, now you can test and execute only a portion of the SQL script when creating or updating an Incorta SQL View, Incorta SQL Table, custom SQL table in the Schema Wizard, or SQL-based data source.

For custom SQL tables, make sure to execute the final script you want before saving as the Schema Wizard will use only the portion you execute to create the custom SQL table. As for Incorta SQL Views, Incorta SQL Tables, and SQL-based data sources, keep only the script you want for creating each.

MV Assistant: recommended values for better MV performance

The MV Assistant helps you identify the appropriate value combination of Spark-related configurations for each materialized view (MV). A dedicated dashboard displays the recommended values and a comparison between the performance of the current and recommended configurations. In addition, the metadata database saves these recommended values per MV. As a schema developer, you can manually apply the recommended values per MV.

The Cloud Admin has to enable the new feature, define the related properties in the Tenant Configurations, and import the MV Assistant dashboard and underlying schemas. Contact Incorta Support to get the required assets.

Note

The MV Assistant is a preview feature in this release.

For more information about the available properties, refer to Tenant Configurations → MV Assistant.

Performance enhancement: Parallel column reading

Incorta can now read parquet segments and loads them into engine memory in parallel to speed up column reading, especially string columns.

This feature enhances the performance of the following:

Creating the PK-index during a load job
Loading data into Loader memory and Analytics memory
Reading columns when creating joins and formula columns
Creating insights and rendering dashboards
Warming up the Loader and Analytics services

Note

This feature is disabled by default. To enable it, contact Incorta Support to edit the engine.properties file in both the Analytics Service and the Loader Service nodes and set both the store.compressed_parallel and parquet.ignore_duplicates properties to true.

Support for Spark R

Starting with this release, Incorta Cloud supports Spark R for creating materialized views (MVs).

2022.9.1 Maintenance Pack

Issue resolved	Area
Schema loads would stall out and the Loader Service became unresponsive during load jobs.	Physical Schemas
Some insights that reference business schema formula columns would fail to render.	Dashboards
Emailed dashboards were not displayed correctly in Outlook 2016.	Dashboards

2022.9.0 Enhancements and Fixes

In addition to the new features and major enhancements mentioned above, this release introduces some additional enhancements and fixes that help to make Incorta more stable, engaging, and reliable.

Issue resolved	Area
A timezone error could occur when attempting to save a draft version of a schema.	Physical Schemas
Improved performance when retrieving model and schema updates	Physical Schemas
An error message will now be displayed when attempting to use unsupported data types in a materialized view.	Physical Schemas
Schema loads could freeze when loading multiple schemas.	Physical Schemas
Filtering joins with the “Last Version” operator would cause the filter validation to never complete.	Physical Schemas
An error would occur when previewing data from a physical schema table in an imported schema.	Physical Schemas
An error would prevent a user from deleting a schema.	Physical Schemas
Deleting tables with certain joins would not actually delete the table and related joins.	Physical Schemas
Updating MV scripts did not update the corresponding notebook script.	Physical Schemas
Query execution failure using the add_months() function referenced from an alias.	Physical Schemas
Non Optimized tables displaying zero rows on the schema status page after a full load.	Physical Schemas
The Preview Data button was not updating to clickable on a loaded table.	Physical Schemas
Business schema views would fallback to spark when they should be served from the engine.	Business Schemas
An aggregated table would encounter an error with the Merge Columns option enabled.	Dashboards
An error would occur when using runtime filters in a dashboard that included an insight based on an SQL view.	Dashboards
Exported PDF versions of insights could have misaligned columns.	Dashboards
A schema sync could become blocked from completion during a dashboard query.	Analyzer
The selected color palette for insight was not displaying as expected.	Visualizations
Mapbox insight drilldowns were not functioning.	Visualizations
The row count of Pivot Tables would display an incorrect result when disabling the Column Grand Total property.	Visualizations
Bar and column charts would not display properly.	Visualizations
Certain configurations of the ServiceNow connector could result in a request failure if the response took too long.	Connectors
A Set prefetch direction toggle has been added to the Custom SQL connector properties. When enabled using a data source that allows this configuration the fetch direction will be set to forward.	Connectors
The Fusion connector would not properly resolve empty or corrupted datasets during incremental loads.	Connectors
The Box connector would create inferred empty columns from Excel files.	Connectors
The Cassandra connector did not use the designated page size.	Connectors
The Service Now connector could timeout and fail extraction.	Connectors
The Inspector tool did not detect columns with non-ASCII characters.	Inspector
Running a query failed and resulted in an `arrayindexoutofbound` exception.	SQL Interface (SQLi)
Enhanced the validation of the tenant password field during Tenant creation in the CMC. Now the tenant password should be at least five characters.	Security
The “last used columns” in the warm-up loader of the CMC would not load tables.	Loader Service
After an upgrade, the data destinations list would not display.	Data Destinations
Users could not install a data application for certain tenants.	Data Applications
Attempting to restore a cluster backup resulted in an error.	Cluster Management Console (CMC)
Export folders in the tenant configurations settings were not being added to the available export list.	Cluster Management Console (CMC)
Cloning a cluster in the cloud portal would never resolve.	Cloud Admin Portal
An incorrect error message appeared when installing a new or updated data application.	Cloud Admin Portal

Known issues

The following are the known issues in 2022.9.0.

Aggregated tables with hierarchical data do not support conditional formatting based on other measures when adding multiple attributes.
When the Pause Scheduled Jobs option is enabled in the Cluster Management Console (CMC), the emails of manually activated scheduled jobs are not sent.
Scheduled jobs of overwritten dashboards and physical schemas are not deleted.
The “You have successfully deleted 1 schema(s)” message is erroneously displayed when trying to delete a physical schema while loading it.
Access rights to a parent folder are not propagated to child folders and dashboards if the same access rights were revoked before.
Dashboard names with special characters will not be displayed correctly when sending the dashboard via email.

For information about all the known issues and workarounds in Incorta Cloud releases, refer to Known Issues in 2022 Releases.