1. Home
  2. Guides
  3. Concepts
  4. Tools
  5. References
  6. Integrations
  7. Data applications
  8. Release Notes
    1. 2024.1 Releases
    2. 2024.7 Releases
      1. 2024.7.x Releases
      2. 2024.7.2 Release
      3. 2024.7.3 Release
      4. 2024.7.4 Release
    3. Cloud Platform Updates
    4. Known Issues
  9. Incorta Premium

Release Notes Incorta 2024.7.2

Release Highlights

2024.7.2 builds on the 2024.7 release with new product capabilities and several enhancements to existing features.

Here are some of the key highlights:

  • An offering for Incorta Premium to unlock advanced features
  • Enhancements to Incorta Copilot
  • Query functions with limits
  • New Public API endpoints

Upgrade Considerations

Important

Upgrade considerations for previous 2024.7.x releases also apply to subsequent 2024.7 releases unless stated otherwise.

After upgrading to 2024.7.2 and with the introduction of Incorta Premium, the following features will be impacted:

  • Notebook for business users: Enable Incorta Premium to be able to access existing business notebooks and create new ones.
  • Spark SQL Views: To create new, explore, edit, and visualize Spark SQL Views, enable Incorta Premium and Spark SQL View in the Server Configurations > Incorta Labs.
  • Data Studio: To avail the Data Studio tab in the Analytics platform, enable Incorta Premium, enable Data Studio for Cloud clusters in the Cloud Admin Portal > Configurations, and enable and configure it in the CMC > Server Configurations > Incorta Data Studio.
  • Copilot: To have access to Incorta Copilot and generative AI capabilities, enable Incorta Premium, enable and configure Copilot for Cloud clusters in the Cloud Admin Portal > Configurations, and enable and configure it in the CMC > Server Configurations > Incorta Copilot.
Semantic Search in Business Schemas

Semantic search within the Business Schemas list won’t be supported in 2024.7.2 even after enabling the Incorta Premium and Copilot.

Data Agent Considerations

The 2024.7.2 release uses the Data Agent version 9.2.1. Please upgrade to the required version.

New Features

In this maintenance pack:

Incorta Premium

With this release, we are happy to package some of our most newest innovative features into the Incorta Premium Package. Features packaged as a part of Incorta Premium include:

  • The Incorta Copilot - The platform integrates generative AI experiences, collectively known as "Copilot," across various interfaces to enhance user interactions. These include assistants for business queries, dashboards, story generation, metadata, notebooks, and data studio, all using natural language to simplify tasks and generate insights.
  • The Incorta Data Studio - The Incorta Data Studio is a new suite enabling Schema Managers to accelerate Materialized View development through Dataflows—low-code workflows made of interconnected Recipes. Users can perform data quality checks, blending, preparation, analysis, and complex operations with Python, SQL, and LLM Recipes, ultimately deploying the workflow into a Physical Schema.
  • Spark SQL Views - Incorta has now extended its Advanced SQL interface to support building Business views in Spark SQL, enabling complex data transformations, dynamic filtering with variables, and seamless visualization within dashboards.
  • Business Notebooks - The Analyzer Notebook enables analysts to develop in PySpark, Spark SQL, Scala, and Spark R, providing access to verified Business Schema views with row-level security, ensuring secure and governed data analysis.

To explore our Premium features in depth, check Incorta Premium.

Enable Views for Copilot

Incorta increases the security and governance of what views can be queried through the Copilot. Previously, any verified view was enabled to be queried by the Copilot by default. Now, a new setting has been introduced to the Business Schema that enables or disables a view to be queried by the Copilot.

To set a view as eligible for use in Copilot, edit a Business Schema, open a Verified View, select More Options (⋮) -> Enable for Copilot

Important

Views are not enabled for Copilot by Default.

Copilot User Role

This release introduces a new role, the Copilot User Role. This role is required for any user who intends to interact with the Copilot capabilities. Since Copilot experiences are built across multiple experiences, Copilot features will be made available based on their level of access to the platform. For example, Schema Managers can leverage Copilot's natural language to SQL capabilities when building materialized views while an Individual Analyzer can leverage natural language to Insight capabilities.

Enhanced querying with limits in variables and In-Query filters

In this release, Incorta introduces two query functions designed for efficient data retrieval when working with internal session and global variables. These functions enable you to specify a limit on the number of records returned, allowing more targeted querying based on defined conditions.

  • queryWithLimit: Returns a specified number of records from a given column, with the option to apply additional filters to further narrow down the results.
  • queryDistinctWithLimit: Returns a limited set of unique (distinct) values from a specified column, with the ability to apply filters to refine the selection of distinct values. It is supported in In-Query filters as well.

New Public API endpoints

Two new endpoints are now available:

  • Create Catalog Folder (/catalog/folder): Creates a new folder after providing the folder name and the ID of its parent folder.
  • Rename and move Dashboards (/dashboards): Renames a list of dashboards, moves them to other folders, or transfers their ownership.

Enhancements and Fixes

Enhancements

EnhancementArea
Incorta now supports analyzing and querying unstructured data, such as PDFs and MS Word files.Incorta Copilot and Data Studio
A new version of the Kyuubi Hive JDBC Jar is now available for connecting Tableau to Incorta via Advanced SQli. The new Jar introduces an enhanced error messaging, providing clearer feedback when incorrect user credentials are supplied.Advanced SQLi

Fixes

Fixed IssueArea
Comments at the beginning or end of the Spark SQL view code resulted in query failureSpark SQL
Incremental Mode activated in the Save MV Recipe in a Dataflow would fail to deploy a MV to the target schema.Data Studio
Upgrading clusters that used a MySQL 8 metadata database from a 6.0.x or earlier release to 2024.7.x might fail.Metadata Database
After the cleanup job ran and removed the load job tracking data, the Schema Designer displayed 0 rows for non-optimized tables if they did not have any successful load jobs during the retention period, or if all load jobs during this period resulted in 0 rows, although these tables might still have data on disk.Schemas

Patch Releases

2024.7.2-P1

Enhancements

EnhancementArea
More flexible conditions for parallel column reading into memory. Previously, only Parquet files with at least 10 million rows and 4 segments qualified for parallel processing. Now, Incorta reads columns from Parquet files in parallel if they meet any of these criteria, significantly optimizing load times, particularly for clusters using Google Cloud Storage (GCS).Parallel column reading

Fixes

Fixed IssueArea
After upgrading to 2024.7.x. previously configured incremental ingest jobs might fail or result in data inconsistencies at the destination.Intelligent Ingest