Release Notes 2022.1.0
The 2022.1.0 release introduces multiple enhancements and new features. The main focus of this release is to enhance the analytical capabilities, data management, security, resource utilization, integration, and performance of the Incorta Cloud. This release introduces a new version of the Excel Add-in, support for locking the data types of output columns in a multi-source table, enhancements in visualizations and dashboards, and improvements in the Cluster Management Console (CMC) monitoring tool.
In addition, the 2022.1.0 release enhances data security by supporting exporting tenants using a security password, which allows including the sensitive data, such as data source credentials and by restricting access to shared schemas in the Notebook Editor. In this release also, you can invite users to your tenant via email. This release offers multiple enhancements in materialized views (MVs) including the support for referencing global variables in MVs and the option to submit MV Spark jobs in cluster mode.
- Support for date and timestamp data types in case and decode conditional statements
- Geo attribute restricted to Geo tray in Advanced Map visualization
- Data lake connector file exclusion
- Global variable support in MVs
- Notebook enforcement of schema permissions
- Invite external users to your Incorta Analytics tenant via email
- New security password for exporting and importing tenants in the CMC
- MV execution in Cluster Mode
- Data type lock in a multi-source table
- New Tenants and Nodes tabs in the CMC Monitor tool
In addition to other enhancements and fixes.
Here are the details of the new features and enhancements.
decode conditional statements in the Analyzer now support the date and timestamp data types in their signature. For more information, refer to Built-in Functions → case and Built-in Functions → decode.
You can drag and drop columns with Geo data types only to the Geo Attribute tray in Advanced Map Visualizations.
For more information, refer to Visualizations → Advanced Map.
The data lake connectors now have an Exclude table data source property. This property allows you to exclude all files with names that match the specified pattern from the load process.
For more information, refer to Connectors → All → Data Lakes.
In this release, you can reference global variables in materialized views when you use the Notebook Editor or the Query Builder to add or edit the MV script (or incremental script) using any supported language. This allows you to centralize the constant values that you reference in multiple MVs, which saves you time and effort and ensures data consistency across all related MVs.
To reference a global variable in an MV, precede the variable name with a double dollar sign, for example,
$$gvar_name. When referencing string, date, and timestamp global variables in an MV, you must use single or double quotes depending upon the MV script language. For Spark Scala, use double quotes; for other languages, use single quotes, for example
Global variables referenced in MVs are evaluated when validating the MV data source and when loading them. Thus, when you edit the value of a global variable, you must perform a full load of the related MVs.
For more information, refer to Concepts → Global Variable.
Notebooks will now enforce the physical schema access control permissions. In a notebook, you will only be able to read a physical schema table and run queries against it if it is shared with you.
Starting with this release, the SuperUser tenant administrator or a user with the SuperRole in an Incorta cloud cluster can invite external users to have access to the tenant via email. For each valid email address that you specify and does not exist in the cluster, an inactive user account is created and an invitation is sent. The invitation email contains a link that allows an external user to accept the invitation, create the account password, activate the user account, and access the shared tenant. The SuperUser tenant administrator or a user with the SuperRole can manage all pending and expired invitations.
While this feature is enabled by default for trial users, Incorta customers have this feature disabled by default. The SuperUser tenant administrator must enable it per tenant in the CMC → Tenant Configurations → Security.
The SuperUser tenant administrator must properly set the email configurations in the CMC for the respective tenant. In addition, the inviter user must have at least one assigned group.
For more information, refer to Security Manager → Manage Invitations.
This feature targets cloud environments only, whether of customers or trial users. It will cause issues in on-premises environments that use Oracle or Derby for the metadata database.
In this release, Incorta is adding an extra security feature to protect your sensitive data, such as the credentials of your data sources, when you export a tenant using the CMC.
A new step is added when you export a tenant, where you can set a security password to be able to export sensitive data. If you choose to export a tenant without a security password, you will have to re-enter all of your data source credentials after importing the tenant.
For more information, refer to CMC Tenant Manager.
This release supports submitting Spark jobs to the Spark cluster in the
cluster deploy mode. In this deploy mode, when executing Spark jobs (loading an MV, for example), the driver application will be created in the Spark cluster instead of the Incorta cluster where the Loader Service exists. The newly supported Spark configuration will help minimize network latency between the drivers and the executors, reduce the required resources on the Loader Service node, and overcome the Out of Memory (OOM) killer issue.
You can set the deploy mode at the MV level or the server level. The server and MV can have different configurations for the deploy mode:
cluster. If the MV deploy mode is defined, it overrides the server configuration. Otherwise, the server deploy mode is applied.
- To set the deploy mode at the MV level, in the MV Data Source dialog, for the
- To set the deploy mode at the server level, in the CMC → Cluster Configurations → Server Configurations → Spark Integration, for Spark Submit Deployment Mode, select cluster or client.
Starting with this release, a schema designer can lock the data type of one or more output columns in a multi-source table so that the data type will not be automatically cast or updated when validating data sources and inferring the output columns in the following cases:
- Accessing the respective multi-source table in the Table Editor
- Adding a new data source to a multi-source table
- Editing one of the data sources in the Data Source dialog and validating all data sources
- Deleting one data source from a multi-source table with more than two data sources
For more information, refer to Tools → Table Editor → Manage data sources
Incorta is introducing two new tabs in the CMC Monitor tool beside the Infrastructure, which are Tenants and Nodes. These new tabs will enable you to view the utilization graphs for tenants and nodes per cluster. For more information, refer to Tools → CMC Monitor.
Version 16.8 of the Excel Add-in is now available. Here are some key features of the new version:
- A new user interface that is consistent with the Incorta Analyzer
- Support for Aggregated Tables in addition to Listing and Pivot Tables
For more information, refer to External Integrations → Excel Add-in.
- Enhanced the cards shown under the Learn More section on the Home page to be user-friendly. Based on the user role, the most relevant cards are highlighted.
- Fixed an issue in which loading a physical schema after removing columns that were used in joins showed a
Successload status instead of
Finished with errors, even when the Stop Loading on First Error toggle was enabled in the CMC
- Fixed an issue in which schema load failed while trying to read parquet files
- Fixed an issue that caused MVs not to read data lake remote tables
- Enhanced the chart legend and tooltips font size and color
- Enhanced the Pie and Donut chart to increase the chart size, wrap long legend text, and decrease the white space in the visualizations
- Fixed an issue in table visualizations in which gridlines were not maintained when the insight was downloaded to PDF
- Fixed an issue in Advanced Map visualizations in which the drill down option did not work for Heatmap layers
- Fixed an issue in which an error message was displayed when saving an insight that was created using the Liquid Fill Gauge custom visualization from the marketplace
- Fixed an issue in which creating a Line visualization with a large amount of data caused failure in rendering the line insight
- Fixed an issue that caused the total to disappear after filtering a donut chart by selecting a legend item and to appear again in a wrong place after reselecting the legend item
- Fixed an issue in which copying a dashboard duplicated the tabs and insights with the same Globally Unique Identifier (GUID), which caused playing your favorite dashboards not to work properly when both the original and duplicated dashboards existed in the Favorites list
- Fixed an issue in which filtering the dashboard that contained a presentation variable by a valid session variable did not return data
- Fixed an issue where the dashboard failed to render after performing a load that followed removing a column used in a join from the table in use
- Fixed an issue in which canceling or closing a modal dialog, such as the Dashboard Filters Manager, caused a dashboard to refresh and to apply the default bookmark
- Fixed a case where a synchronization job that was started and completed while rendering a related insight caused rendering the insight to fail
- Resolved an issue that caused the upper border of the header row not to be resized when changing the column width
- Fixed a case where adding or editing an insight caused downloading it to Excel to fail
- Fixed an issue in which opening the Query Plan Viewer resulted in an error when using a formula in the Aggregate Filter tray
- Fixed an issue in which data failed to render when you drag a table to the Measure tray
- Fixed an issue in which SQL Generation failed when using a presentation variable in a formula column in Table insights
- Fixed an issue that caused the Analyzer to fail after editing an insight
- Fixed an issue that caused column headers to disappear in a table insight with zero rows when transposing the table
- Fixed an issue that caused the Audit files to be corrupted
- Fixed an issue in which the decimal places were limited to 10 only in PostgreSQL
- Fixed an issue in which the Data Agent version 2.0.1 could not start, resulting in an error
- Fixed an issue in which the
agoanalytic function returned incorrect results when referencing business schema columns
- Resolved an issue in which adding multiple users to a group worked only when users existed on the same page of the list
- Resolved an issue that caused importing a tenant while ignoring the tenant’s properties file to fail