Release Notes 2022.1.0

Release Highlights

The 2022.1.0 release introduces multiple enhancements and new features. The main focus of this release is to enhance the analytical capabilities, data management, security, resource utilization, integration, and performance of the Incorta Cloud. This release introduces a new version of the Excel Add-in, support for locking the data types of output columns in a multi-source table, enhancements in visualizations and dashboards, and improvements in the Cluster Management Console (CMC) monitoring tool.

In addition, the 2022.1.0 release enhances data security by supporting exporting tenants using a security password, which allows including the sensitive data, such as data source credentials and by restricting access to shared schemas in the Notebook Editor. In this release also, you can invite users to your tenant via email. This release offers multiple enhancements in materialized views (MVs) including the support for referencing global variables in MVs and the option to submit MV Spark jobs in cluster mode.

New Features

Here are the details of the new features and enhancements.

Support for date and timestamp data types in CASE and DECODE conditional statements

The case and decode conditional statements in the Analyzer now support the date and timestamp data types in their signature. For more information, refer to Built-in Functions → case and Built-in Functions → decode.

Geo columns restricted to Geo Attributes tray in Advanced Map visualization

You can drag and drop columns with Geo data types only to the Geo Attribute tray in Advanced Map Visualizations.

For more information, refer to Visualizations → Advanced Map.

Data Lake connector file exclusion

The data lake connectors now have an Exclude table data source property. This property allows you to exclude all files with names that match the specified pattern from the load process.

For more information, refer to Connectors → All → Data Lakes.

Global variable support in MVs

In this release, you can reference global variables in materialized views when you use the Notebook Editor or the Query Builder to add or edit the MV script (or incremental script) using any supported language. This allows you to centralize the constant values that you reference in multiple MVs, which saves you time and effort and ensures data consistency across all related MVs.

To reference a global variable in an MV, precede the variable name with a double dollar sign, for example, $$gvar_name. When referencing string, date, and timestamp global variables in an MV, you must use single or double quotes depending upon the MV script language. For Spark Scala, use double quotes; for other languages, use single quotes, for example '$$string_gvar_name'.

Important

Global variables referenced in MVs are evaluated when validating the MV data source and when loading them. Thus, when you edit the value of a global variable, you must perform a full load of the related MVs.

For more information, refer to Concepts → Global Variable.

Notebook enforcement of physical schema permissions

Notebooks will now enforce the physical schema access control permissions. In a notebook, you will only be able to read a physical schema table and run queries against it if it is shared with you.

Invite external users to your Incorta Analytics tenant via email

Starting with this release, the Super User tenant administrator or a user with the SuperRole in an Incorta cloud cluster can invite external users to have access to the tenant via email. For each valid email address that you specify and does not exist in the cluster, an inactive user account is created and an invitation is sent. The invitation email contains a link that allows an external user to accept the invitation, create the account password, activate the user account, and access the shared tenant. The Super User tenant administrator or a user with the SuperRole can manage all pending and expired invitations.

While this feature is enabled by default for trial users, Incorta customers have this feature disabled by default. The Super User tenant administrator must enable it per tenant in the CMC → Tenant Configurations → Security.

The Super User tenant administrator must properly set the email configurations in the CMC for the respective tenant. In addition, the inviter user must have at least one assigned group.

For more information, refer to Security Manager → Manage Invitations.

Warning

This feature targets cloud environments only, whether of customers or trial users. It will cause issues in on-premises environments that use Oracle or Derby for the metadata database.

New security password for exporting and importing tenants in the CMC

In this release, Incorta is adding an extra security feature to protect your sensitive data, such as the credentials of your data sources, when you export a tenant using the CMC.

A new step is added when you export a tenant, where you can set a security password to be able to export sensitive data. If you choose to export a tenant without a security password, you will have to re-enter all of your data source credentials after importing the tenant.

For more information, refer to CMC Tenant Manager.

MV execution in Cluster Mode

This release supports submitting Spark jobs to the Spark cluster in the cluster deploy mode. In this deploy mode, when executing Spark jobs (loading an MV, for example), the driver application will be created in the Spark cluster instead of the Incorta cluster where the Loader Service exists. The newly supported Spark configuration will help minimize network latency between the drivers and the executors, reduce the required resources on the Loader Service node, and overcome the Out of Memory (OOM) killer issue.

You can set the deploy mode at the MV level or the server level. The server and MV can have different configurations for the deploy mode: client or cluster. If the MV deploy mode is defined, it overrides the server configuration. Otherwise, the server deploy mode is applied.

To set the deploy mode at the MV level, in the MV Data Source dialog, for the spark.submit.deployMode property, enter cluster or client.
To set the deploy mode at the server level, in the CMC → Cluster Configurations → Server Configurations → Spark Integration, for Spark Submit Deployment Mode, select cluster or client.

Data type lock in a multi-source table

Starting with this release, a schema designer can lock the data type of one or more output columns in a multi-source table so that the data type will not be automatically cast or updated when validating data sources and inferring the output columns in the following cases:

Accessing the respective multi-source table in the Table Editor
Adding a new data source to a multi-source table
Editing one of the data sources in the Data Source dialog and validating all data sources
Deleting one data source from a multi-source table with more than two data sources

For more information, refer to Tools → Table Editor → Manage data sources

New Tenants and Nodes tabs in the CMC Monitor tool

Incorta is introducing two new tabs in the CMC Monitor tool beside the Infrastructure, which are Tenants and Nodes. These new tabs will enable you to view the utilization graphs for tenants and nodes per cluster. For more information, refer to Tools → CMC Monitor.

A new version of the Excel Add-in

Version 16.8 of the Excel Add-in is now available. Here are some key features of the new version:

A new user interface that is consistent with the Incorta Analyzer
Support for Aggregated Tables in addition to Listing and Pivot Tables

For more information, refer to External Integrations → Excel Add-in.

Additional Enhancements and Fixes

Home Page

Enhanced the cards shown under the Learn More section on the Home page to be user-friendly. Based on the user role, the most relevant cards are highlighted.

Physical Schema

Fixed an issue in which loading a physical schema after removing columns that were used in joins showed a Success load status instead of Finished with errors, even when the Stop Loading on First Error toggle was enabled in the CMC
Fixed an issue in which schema load failed while trying to read parquet files
Fixed an issue that caused MVs not to read data lake remote tables

Visualizations

Enhanced the chart legend and tooltips font size and color
Enhanced the Pie and Donut chart to increase the chart size, wrap long legend text, and decrease the white space in the visualizations
Fixed an issue in table visualizations in which gridlines were not maintained when the insight was downloaded to PDF
Fixed an issue in Advanced Map visualizations in which the drill down option did not work for Heatmap layers
Fixed an issue in which an error message was displayed when saving an insight that was created using the Liquid Fill Gauge custom visualization from the marketplace
Fixed an issue in which creating a Line visualization with a large amount of data caused failure in rendering the line insight
Fixed an issue that caused the total to disappear after filtering a donut chart by selecting a legend item and to appear again in a wrong place after reselecting the legend item

Dashboards

Fixed an issue in which copying a dashboard duplicated the tabs and insights with the same Globally Unique Identifier (GUID), which caused playing your favorite dashboards not to work properly when both the original and duplicated dashboards existed in the Favorites list
Fixed an issue in which filtering the dashboard that contained a presentation variable by a valid session variable did not return data
Fixed an issue where the dashboard failed to render after performing a load that followed removing a column used in a join from the table in use
Fixed an issue in which canceling or closing a modal dialog, such as the Dashboard Filters Manager, caused a dashboard to refresh and to apply the default bookmark
Fixed a case where a synchronization job that was started and completed while rendering a related insight caused rendering the insight to fail
Resolved an issue that caused the upper border of the header row not to be resized when changing the column width
Fixed a case where adding or editing an insight caused downloading it to Excel to fail

Analyzer

Fixed an issue in which opening the Query Plan Viewer resulted in an error when using a formula in the Aggregate Filter tray
Fixed an issue in which data failed to render when you drag a table to the Measure tray
Fixed an issue in which SQL Generation failed when using a presentation variable in a formula column in Table insights
Fixed an issue that caused the Analyzer to fail after editing an insight
Fixed an issue that caused column headers to disappear in a table insight with zero rows when transposing the table

Materialized Views and SQLi

Fixed an issue that caused the Audit files to be corrupted
Fixed an issue in which the decimal places were limited to 10 only in PostgreSQL

Data Agent

Fixed an issue in which the Data Agent version 2.0.1 could not start, resulting in an error

Miscellaneous

Fixed an issue in which the ago analytic function returned incorrect results when referencing business schema columns
Resolved an issue in which adding multiple users to a group worked only when users existed on the same page of the list
Resolved an issue that caused importing a tenant while ignoring the tenant’s properties file to fail