What's New in Incorta Cloud 2022.9.0

Release Highlights

Incorta is introducing multiple enhancements in this release. You can now format a measure based on the value of another measure. Parallel column reading greatly enhances the performance of data loading and dashboard rendering. In addition, our new MV Assistant can recommend Spark settings to better enhance materialized view (MV) loading time and performance.

Additional enhancements include tracking the dependencies of formula columns in physical schema tables and MVs, coloring data in Tag Cloud insights according to other measures, and testing and executing a portion of the SQL script of Incorta SQL Views and Tables, custom SQL tables, and SQL-based data sources. A new data application marketplace is available to paid accounts upon subscription and to all trial users.

New Features

• Conditional formatting based on another measure
• Support Color By in Tag Cloud visualization
• Data Applications Marketplace
• Tracking formula column dependencies
• Executing a portion of an SQL script
• MV Assistant: recommended values for better MV performance
• Performance enhancement: Parallel column reading
• Support for Spark R

Conditional formatting based on another measure

For chart and table insights that support conditional formatting (Bar, Column, KPI, Listing Table, Aggregated Table, and Pivot Table insights), you can apply different formats on a measure (column or formula) depending on the value or the aggregated value of another column or a formula. You can add multiple conditional formatting rules to the same measure and arrange them as required. If multiple conditions overlap, the first rule in order prevails.

If the measure column and the conditional formatting base column exist in different objects, make sure that there is a valid query plan between the two objects.

Here is an example of formatting a measure based on another in a column chart.

• In the Navigation bar, select Content, and then select a dashboard or create a new one.
• Create a new column insight, and then select the data set you want to use. The example uses the Sales schema.
• Drag a column to the Grouping Dimension tray and another to the Measure tray. For example, add Category from the Products table and Revenue from the Sales table, respectively.
• Expand the Revenue pill, and then under Conditional Formatting, drag a function to the Based On placeholder. In the Formula Builder, enter the formula you want, for example, sum(SALES.SALES.AMOUNT_SOLD) - sum(SALES.SALES.COST_OF_GOODS).
• Expand the formula pill, change the Column Title to Profit and the Aggregation to Sum.
• Select Add Condition. You can add multiple conditions as required. For each condition, notice the message that denotes the conditional formatting base column and the aggregation used. select the Operator, enter the threshold Value, and finally set the Background and Text Color when the condition is true.
• Drag and drop the condition pills to determine the order of applying the conditions.
• Save the insight.

Viewing a chart insight as an aggregated table and downloading or sending the insight as PDF respects applied conditional formats based on other measures. However, downloading or sending the insight as an Excel file ignores these conditional formats.

Support Color By in Tag Cloud Visualization

The Tag Cloud visualization now has the Color By tray which enables you to use coloring as an additional dimension to view your data.

In the Analyzer, select the Tag Cloud visualization. In the following example, we are measuring the revenue based on categories and colored by subcategories.

From the Data panel, you can add a column or a formula to the Color By tray. The text in the chart is grouped by the column added to the Color by tray into colored groups.

The Color by pill is a filterable column for insight. A defined coloring dimension introduces a Filter by menu as a user interaction for the tag cloud chart.

For more information, refer to Visualizations → Tag Cloud.

Data Applications Marketplace

A new data applications marketplace is now available to view and download different data applications that either Incorta or Incorta partners provide. You will also be able to upload and share your own data applications as an Incorta Partner.

After signing in to your Incorta Cloud cluster, select the Marketplace tab.

By default, you will find a list of all available data applications. The list is displayed as cards. Each card has the following details:

• Business function name
• Data application name
• Provider name
• Brief description about the data application

You can filter this list using different aspects, for example, you can filter based on Data Source and Business Function types. You can also search the data applications by entering any relevant keyword in the search bar, such as data application name, source, business function, and provider.

In this list, you will be able to see popular, recommended, and new data applications tagged so you can easily identify them.

You will also be able to select Submit a Request to share any ideas for data applications that you did not find in the list.

Also, you can select Become an Incorta Partner to upload and share your own data application as a partner to the marketplace.

When you select a data application, you can view its overview, installation requirements, and relevant resources from documentation and community articles.

To install the selected data application, select the button Install automatically or Contact to Install to send an email to Incorta to install the data application on your cloud cluster.

Tracking formula column dependencies

You can now access the Data Lineage Viewer of formula columns in physical schema tables and materialized views (MVs). Track the dependencies of these formula columns and preview the list and diagram of their dependent entities.

Here are the steps to preview the dependencies of a formula column in a physical schema table or MV.

• In the Navigation bar, select Schema, select a physical schema, and then select the physical schema table or MV you want.
• In the Table Editor, on the Columns tab, in the Formula Columns section, hover over the formula column you want.
• Select the DNA icon.
• Use the Data Lineage Viewer to review the list and diagram of dependent entities.

Executing a portion of an SQL script

To facilitate the iterative process of writing SQL in Incorta, you can now test and execute only a portion of the SQL script when creating or updating an Incorta SQL View, Incorta SQL Table, custom SQL table in the Schema Wizard, or SQL-based data source.

For custom SQL tables, make sure to execute the final script you want before saving as the Schema Wizard will use only the portion you execute to create the custom SQL table. As for Incorta SQL Views, Incorta SQL Tables, and SQL-based data sources; keep only the script you want for creating each.

To test or execute a portion of the SQL script or query, in the Query Editor, select the portion you want, and then select Execute.

MV Assistant: recommended values for better MV performance

The MV Assistant helps you identify the appropriate value combination of Spark-related configurations for each materialized view (MV). A dedicated dashboard displays the recommended values and a comparison between the performance of the current and recommended configurations. In addition, the metadata database saves these recommended values per MV. As a schema developer, you can manually apply the recommended values per MV.

The Cloud Admin has to enable the new feature, define the related properties in the Tenant Configurations, and import the MV Assistant dashboard and underlying schemas provided by the Incorta Support team.

Here are the steps to enable and configure the MV Assistant.

• Sign in to the Cluster Management Console (CMC) as the Cloud Admin user.
• In the Navigation bar, select Clusters, and then select your cluster.
• To enable and configure the MV Assistant for a specific tenant, select the Tenants tab, and then select Configure for your tenant. To enable and configure it for the default tenant, select the Cluster Configurations tab, and then the Default Tenant Configurations.
• In the configurations pane, select MV Assistant.
• Enable the MV Assistant toggle and set the required properties.

For more information about the available properties, refer to Tenant Configurations → MV Assistant.

Performance enhancement: Parallel column reading

Incorta is enhancing the performance of data loading, dashboard rendering, and service warmup. Incorta can now read parquet segments and loads them into engine memory in parallel to speed up column reading, especially string columns.

This feature enhances the performance of the following:

• Creating the PK-index during a load job
• Loading data into Loader memory and Analytics memory
• Reading columns when creating joins and formula columns
• Creating insights and rendering dashboards
• Warming up the Loader and Analytics services

Support for Spark R

Starting with this release, Incorta Cloud supports Spark R for creating materialized views (MVs).