Release Notes 2024.1.x

Release Highlights

In the 2024.1.x release, Incorta is introducing multiple new features that aim to enhance and ease the user experience:

• Introducing the Incorta Copilot, an OpenAI/ChatGPT integration that brings the power of generative AI to various parts of the platform, including generating dashboard insights using natural language.
• Introducing Incorta Data Studio, a low-code data enrichment tool available as a public preview. With the data Data Studio, users can add, connect, and edit 15+ recipe types and deploy to pySpark materialized views without writing code.
• Introducing a new advanced SQL interface as an alternative to the existing SQL-Interface that is fully Spark SQL compliant, more performant, and provides improved integration with external analytical tools
• A standalone Notebook IDE for data teams as a public preview feature.
• A new Connectors marketplace is now available. With the connectors marketplace, you can install, upgrade, or downgrade connectors without waiting for releases.
• More enhancements to dashboard free-form layouts to help build visually appealing dashboards.

The 2024.1.1, 2024.1.2, and 2024.1.3 maintenance packs include additional features and enhancements.

Upgrade considerations

Caching mechanism enhancements

The caching mechanism in Analyzer views for dashboards has been enhanced by caching all columns in the view to prevent query inconsistencies. To optimize performance and reduce off-heap memory usage, creating views with only the essential columns used in your dashboards is recommended.

Schema names

Starting with this release, the names of newly created schemas and business schemas are not case-sensitive. Therefore, you cannot create a new schema or business schema that shares the same name with existing schemas or business schemas using different letter cases. To maintain backward compatibility, the names of existing schemas and business schemas will not be affected.

Lookup function validation

A new validation is now applied to the lookup() function that prevents saving an expression where the result lookup field and primary key field N parameters are not from the same object. As a result, existing expressions with a lookup() function must be updated accordingly; otherwise, these expressions will return #ERROR.

Like function fix

As a result of fixing an issue with the like() function, you must load all tables that use this function from staging.

Custom SQL Connectors

Starting this release, Incorta has changed the directory used for the custom SQL connectors. After upgrading to this release, you must contact Incorta Support to assist you with moving the custom SQL connectors you have configured previously to the appropriate directory. In case of using a Data Agent, you must also copy the custom SQL JDBC driver jars to incorta.dataagent/extensions/connectors/shared-libs/ on the Data Agent host.

New Features

Dashboards, Visualizations, and Analytics

• Dashboard comments collaboration
• Group insights in free-form layout
• Summary Component
• Shapes Components
• Dashboard presentation mode
• Sharing dashboard to Slack and Microsoft Teams
• Null handling enhancements
• Notebook for Analyze Users

Data Management Layer

• Incorta Data Studio
• Connectors marketplace
• Oracle Cloud Applications V2 connector
• GraphQL connector
• Netsuite Searches connector
• OneDrive connector
• Kyuubi connector
• MariaDB connector
• Log-based incremental load
• Incorta Data Delivery enhancements
• Azure Synapse data delivery enhancement
• Date and number format at the view column level
• SQL-compliant verified Business Schema Views
• New Public API endpoints
• New Date functions

Incorta Copilot

• Incorta Copilot Configuration
• Copilot for dashboard
• Copilot for Notebooks
• Copilot for Data Studio
• AI-supported features for business schemas
  • Semantic search for view columns

Architecture and Application Layer

• Enhancements to data load notifications
• Retention of load job tracking data
• Load Plan DAG Viewer enhancements
• Exporting and importing load plans
• Default schema load type at the load plan level
• Load Job Details Viewer enhancements
• Support for merging Parquet segments during loading from staging
• Support for manual load plan execution
• Enhanced the responsiveness of Post-load interruption
• Pause Scheduled Jobs enhancements
• Support OAuth for JDBC connection
• Monitoring the file system usage
• Enhanced performance with parallel snapshot reading

SQLi and Integrations

• Advanced SQL Interface

Dashboard comments collaboration

Introducing the new dashboard comments feature, where you can collaborate and communicate with other users by commenting on available dashboards.

Using this feature, you will be able to:

• Post a comment.
• Reply to comment.
• Edit a comment.
• Delete a comment.
• Mention other users.
• Get notified via email in the following cases:
  • Another user mentions you in a comment.
  • A comment you are mentioned in is edited.
  • A reply is posted to a thread you are following.
  • Users reply to your comments.

If a dashboard has a comment, you will find a red dot on the chat bubble icon; knowing that the number next to the bubble indicates the number of unread comments.

For more information, refer to Tools → Dashboard Manager.

Group insights in free-form layout

Now you can put several insights in a group in the free-form layout, which facilitates the user experience in managing these insights by either bringing them forward or putting them back.

You can always add insights to an already existing group by dragging and dropping this insight into the group.

For more information, refer to Tools → Dashboard Manager.

Summary Component

In this release, Incorta is introducing a new smart Summary component that when added to a dashboard tab, it summarizes all insights within this tab.

The Summary component is a native component that does not require any AI integration.** **You can find the Summary component under the Others category in the Add Insight panel.

Shapes Components

With the 2024.1.0 release, Incorta is enabling you to elevate dashboard designs with the new shapes and icons feature.

You can now integrate lines as delimiters between insights, creating a clear and organized visual hierarchy. Use rectangles strategically to differentiate between various insights.

With this addition, you can incorporate lines, arrows, rectangles, circles, and a variety of other shapes and icons into your dashboard creations.

You can also embed text within your added shapes, resize them, add shadows, and change icon styles.

You can find the new Shapes category in the Add Insight panel with the following available:

• Rectangle
• Circle
• Line
• Arrow
• Icon
• Text

Dashboard Presentation Mode

Now you can present your dashboard directly through the new dashboard presentation mode introduced in this release. A new presentation icon is added in the action bar in the Dashboard Manager that enables you to show your dashboard tabs in full-screen mode.

Sharing dashboard to Slack and Microsoft Teams

In this release, Incorta is enabling you to send dashboards using messages to Slack and Microsoft Teams (MS Teams). You can send dashboards to both private and public.

As a CMC admin, you must first configure Slack and MS Teams to enable Incorta users to share dashboards using these communication platforms.

To configure Slack, refer to the Slack Integration document.

To configure MS Teams, refer to the Microsoft Teams Integration document.

Null handling enhancements

The Analytics Service has extended its support for null value handling in the following areas:

• Functions:
  • Arithmetic Functions
  • Boolean Functions
  • Conversion Functions
  • Date Functions
  • Miscellaneous Functions
  • Analytics Functions
• Filtering data based on formula columns.
• Sorting based on formula columns.
• Formulas involving logical and comparison operators.
• Arithmetic operations.

This expansion of support for null value handling in these areas signifies an important improvement in the capabilities of the Analytics Service, allowing for more comprehensive and accurate data analysis and manipulation.

For more information, refer to References → Null Handling.

Notebook for Analyze Users

Starting this release, Incorta is introducing the Notebook for users with Analyze Users user role can use the notebook within the Content Manager. The Notebook for Analyze Users only supports verified Views.

Known limitations

• You cannot create or import Notebooks in folders.
• Sharing Notebooks is currently not available.
• Only 10 concurrent connections are allowed per cluster to create Notebooks simultaneously. Knowing that if a user has two open tabs for Notebook, it is considered two different sessions.

Incorta Data Studio

Incorta is adding the new Data Studio tool to its bundle this release. Data Studio is a powerful tool that enables users with minimal technical expertise to create Materialized Views (MVs) using a simple drag-and-drop interface.

The Data Studio is disabled by default, you must contact Incorta Support to have it enabled in your cluster.

Using the Data Studio, schema managers can perform multiple actions and transformations on your data using sequential steps and recipes. You use your raw data and apply a series of steps and pre-built recipes to transform your data and create MVs.

Known Limitations

• Any change in the MV from a physical schema will not reflect in the data studio.
• While using a certain schema, Incorta locks that schema preventing any updates.
• Users who do not have access to schemas will have view-only access to data flows where these schemas are used.
• Deleting users will subsequently delete any data flows they have created.

Connectors marketplace

Incorta is introducing the new Connectors marketplace to the cloud users. With the new marketplace, you can install, upgrade, and downgrade your connector version independently from any Incorta releases. In addition, there is no longer downtime to have a connector installed, upgraded, or downgraded and hence you no longer need to restart your services.

For more information, refer to References → Connectors.

Oracle Cloud Applications V2 connector

Incorta is introducing a new enhanced and improved version of the Oracle Cloud Application connector, where you can control who triggers the BICC jobs (BICC or Incorta) and avoid any synchronization issues that may occur.

For more information, refer to Connectors → Oracle Cloud Applications V2.

GraphQL connector

Incorta is introducing the new GraphQL connector in this release. GraphQL is a query language for APIs and a runtime for fulfilling queries with existing data. The GraphQL connector uses the cdata.jdbc.graphql.jar driver to connect to a GraphQL resource and get data.

For more information, refer to Connectors → GraphQL.

NetSuite Searches connector

Incorta added a new connector for NetSuite Saved Searches in this release. NetSuite enables users to search for any record type in the system and save this search in the form of a Saved Search. Incorta connects to retrieve this data to be able to process it and build insights.

For more information, refer to Connectors → NetSuite Searches.

OneDrive connector

The OneDrive connectors enable you to connect to your Microsoft OneDrive. The OneDrive cloud service connects you to all your files stored in the cloud. It lets you store and protect your files, share them with others, and get to them from anywhere on all your devices.

For more information, refer to Connectors → Microsoft OneDrive.

Kyuubi connector

The new connector is a JDBC connector that enables you to connect to Apache Kyuubi. Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on Data Warehouses and Lakehouses. Kyuubi builds distributed SQL query engines on top of various kinds of modern computing frameworks.

For more information, refer to Connectors → Kyuubi.

MariaDB connector

In this release, Incorta is introducing the new MariaDB connector. MariaDB is one of the popular open-source relational databases. It is also the default database in most Linux distributions.

For more information, refer to Connectors → MariaDB.

Log-based Incremental Load

Incorta is now supporting log-based incremental load using the change data capture (CDC). CDC is the process of identifying and capturing changes made to data in a database using logs and then delivering those changes in real-time to a downstream process or system.

Known limitations

• For the time being, Incorta does not track deletion updates through the log-based incremental load (CDC).
• In case of deleting records and performing full load, make sure to reconfigure the Debezium connector to avoid retrieving deleted records from the snapshot; knowing that this limitation is resolved starting connector version 2.0.2.2.
• Minimal mismatch in column INTERVAL data types.
• This feature supports Kafka topics that use a single partition only.

For more information, refer to the What's new in 2024.1.0.

Incorta Data Delivery enhancements

Now you can choose which tables to send to your data destination. While setting up a data destination for a schema, you find a new section that is collapsed by default called Tables. Using this section, you can configure which tables you want to send to the data destination during the load process.

Through the Tables section, you can filter tables by entering a keyword in the search bar. You will also find the Show only selected toggle that shows only selected tables when enabled.

Azure Synapse data delivery enhancement

For Microsoft Azure Synapse, Incorta now discovers and stores a column string length in the metadata. Hence, when you load your data and send it to the destination, the data is sent based on the discovered column length if it is available, otherwise, Incorta will use the Default String Length you have previously configured.

After upgrading to this release, Incorta will discover a column string length with the next full ingest you perform.

Known limitation:

There are cases where column length cannot be discovered, such as calculated columns or columns of data sources based on text files such as Excel and CSV files.

Date and number format at the view column level

Now, you can set the date and number formats per column or formula column in a business schema view. Once configured, this format will be the column’s default format whenever the column or formula column is added to an insight. However, you can still apply a different format to the insight column. You can also update columns in existing insights to inherit the view column format.

Notes:

• After upgrading to 2024.1.0, updating the view column format will not impact existing insight columns unless you configure insight columns to inherit the view column format.
• Changing the view column format affects all insights where the insight column inherits the view column format.
• Format updates at the view column level will not override the format specified at the insight column level.
• If no format is specified at the insight or view levels, the format will be as follows:
  • Date columns: Short Date
  • Timestamp columns: no format
  • Numeric columns: no format, except for decimal columns
• You cannot set the default format for columns in Incorta Analyzer or SQL views.

SQL-compliant verified Business Schema Views

The Analytics Engine can now validate business views for compatibility with external BI tools and SQL compliance. This enhancement will empower you to make more informed decisions when selecting business views for your analytics and reporting needs. You can easily identify a verified business view by the green icon displayed next to it in the Business Schema Designer.

A Verified view is a business schema view that:

• Does not contain formulas referencing columns from Analyzer Views or SQL Views.
• Does not contain aggregation functions.
• Has a valid query plan, which means any of the following:
  • A base table is explicitly defined and is valid for all the view columns (constitutes a valid join path).
  • If there is no defined explicit base table, there must be a valid implicit base table connecting all columns in the view.

If the view does not meet any of the above criteria, it will not be considered verified. You can view the reason why it is not verified by hovering over the gray icon and clicking Show Details.

Current limitations and known issues

• This feature applies only to business schema views.
• Analyzer and SQL views are excluded from the verification process.
• If a view is considered unverified due to multiple reasons, only the first encountered one will be displayed.
• If a business schema view contains only session or global variables, it is considered unverified.

New Public API endpoints

This release introduces the following new endpoints:

Content Manager (Catalog) endpoints

• Search Catalog (catalog/search): searches for dashboards and folders that a user owns or has access to in the Content Manager (Catalog) and its subfolders by a defined keyword.
• List Catalog Content (catalog): lists folders and dashboards that a user owns or has access to in the root directory of the Content Manager (Catalog).
• List Catalog Folder Content (catalog/folder/{folderId}): lists folders and dashboards that a user owns or has access to in a specific folder in the Content Manager (Catalog).

Load plan execution and status endpoints

• Execute Load Plan (/load-plan/execute): executes a specific load plan and returns the load plan execution ID.
• Load Plan Execution Status (load-plan/status/{execution-id}): returns the status of a load plan execution, with or without the details per load group, schema, or object.

New Date functions

This release introduces the following new Date functions:

• yearQuarter(date_timestamp exp): Returns an integer that represents the year and quarter of a given date or timestamp column or expression. In the returned value, the first four digits represent the year while the last two digits represent the quarter. Example: this function, yearQuarter(date("2023-10-25")), returns 202304.
• yearQuarter(): Returns an integer that represents the current year and quarter. In the returned value, the first four digits represent the year while the last two digits represent the quarter. Example: 202401.
• yearMonth(date_timestamp exp): Returns an integer that represents the year and month of a given date or timestamp column or expression. In the returned value, the first four digits represent the year while the last two digits represent the month. Example: this function, yearMonth(date("2023-10-25")), returns 202310.
• yearMonth(): Returns an integer that represents the current year and month. In the returned value, the first four digits represent the year while the last two digits represent the month. Example: 202401.

Incorta Copilot Configuration

Incorta Copilot is the new AI feature that integrates with several Incorta modules and enables users to analyze data, build insights, and write code blocks simply by asking questions in natural language. 

Incorta supports the integration with OpenAI, the copilot is currently integrated within the Analyzer, Dashboards, Data Studio, Business Schema, and Notebook.

Currently, the copilot is a preview feature, to use it please contact the Incorta Support.

After Incorta avails the copilot for you, the CMC admin must log in to the CMC enable the Copilot, and provide the information needed to complete the integration.

Copilot for Dashboard

Copilot in dashboards enables users to directly query the data they have access to and find relevant insights in existing dashboards. Incorta Copilot in dashboards can suggest questions to help you understand the data and uncover new insights. 

You will be able to use the copilot in dashboards using the Copilot button in the header menu. When you select it, it opens a chat window where you can type in your questions and hence get your answers.

Copilot for Notebooks

The copilot also exists in Notebooks, where it can help you debug your Notebook code, or even write it. Start typing in your questions within the Notebook paragraph.

Copilot for Data Studio

Copilot for Data Studio exists in multiple recipes that can help you achieve your goal by typing in a description to enter your code or query statement.

You can use the copilot in the following recipes:

• Custom Code
• Data quality
• New column
• SQL

AI-supported features for business schemas

In this release, you can use the integrated AI tools to generate and suggest descriptions and labels for business schema columns, allowing for:

• Providing more accurate and consistent descriptions and labels.
• Reducing manual effort.
• Ensuring data clarity.

The AI tool creates and suggests column labels based on the column name while it generates and suggests column descriptions based on the table and schema names. You can select one or more columns to generate descriptions or labels for.

Semantic search for view columns

By integrating with Copilot, Incorta has expanded the search capabilities within the business schema list to include business view columns. This enhancement allows for semantic searches for columns across all business schemas on the list, even supporting searches in multiple languages. When searching in the business schema list, the results include both columns that semantically match the search term and business schemas that exactly match it.

Enhancements to data load notifications

This release introduces significant enhancements to data load notifications, including notifications on load plans instead of schemas and a new notification type for jobs that take longer than expected.

Load plan notifications

Now, you can create data load email notifications at the load plan level rather than the schema. The Schema Notifications tab is no longer available and is replaced by the Notifications tab.

• The notifications list will show only notifications that the logged-in user has created. However, the Super User and users with the SuperRole will have full access to all notifications.
• Deleting a load plan deletes its data load notifications.

Upgrade considerations

Existing schema load notifications will be migrated to load plan notifications during the release upgrade process. Each schema load notification will be automatically assigned to all single-schema load plans for this schema. The migration process will not migrate schema notifications to load plans having multiple schemas. Additionally, schemas with no single-schema load plans will not have their notification settings migrated.

• The Last Modified Date of notification will reflect the migration date.
• Migrating a notification will not change its owner.
• Migrated notifications will follow the following naming convention: [notification name]_[load plan name].

Notifications of jobs taking longer than expected

You can also create email notifications for load jobs that take longer than expected, based on the load time of recent load jobs for the same load plan. This feature helps you detect delays in data refresh cycles as early as possible and act accordingly.

Retention of load job tracking data

With high-frequency refresh cycles, the size of load job tracking data, including load job history, load plan executions, and schema update jobs tends to increase over time, which might impact the system’s performance and the metadata database’s volume. That is why Incorta is introducing the Retention period of load job tracking data (In months) option available to the Cluster Management Console (CMC) admins to control the period for which Incorta retains load job tracking data. This option exists under Server Configurations > Tuning, and admins can set the retention period in months. The default is Never, which means that the feature is disabled.

If you enable this feature, a cleanup job runs whenever the Analytics Service starts and every 24 hours afterward and deletes tracking data that exceeds the specified retention period. However, the tracking data of the latest successful load job or schema update job will not be deleted.

Notes and recommendations

• When the cleanup job runs for the first time, it locks the metadata database during the deletion process. The locking duration depends on the number of records that the job will delete.
• It is recommended that you suspend the Scheduler before enabling the feature. Then, start the Analytics Service only and wait for a few minutes before you start the Loader Service.
• It is also recommended that you first configure the feature to start with a long retention period, then change the configuration afterward to a shorter period, and so on until you reach the required retention period. This will reduce the database lock time when the cleanup job runs for the first time.

Load Plan DAG Viewer enhancements

The Load Plan DAG Viewer comes with an enhanced user experience where you can:

• Hide and show task groupings. Grouped nodes show the number of tasks included.
• Search and filter the group panel (which now shows the included schemas and their objects) by the schema or object name. Selecting a schema or object filters the diagram to show only the tasks and nodes for the selected schema or object. Also, it filters the diagram legend to show the related task or node types only.
• Search the diagram by the object name. The search box shows matching objects categorized by task type. The search is limited to the current diagram; therefore, it is affected by the filters applied using the group panel. Selecting an object from the search list highlights the object in the respective node and also highlights it along with its upstream and downstream tasks in all expanded nodes across the diagram, showing the full path of the selected object.

Exporting and importing load plans

To facilitate migrating load plans from one environment to another or one cluster to another, you can now export and import one or more load plans, along with their scheduler details. When importing load plans, you can overwrite existing load plans that share the same name.

• The imported load plan has one or more schemas that do not exist in the target tenant.
• You don’t have edit access rights to all schemas in the load plan.
• The imported load plan shares the same name with an existing load plan while the Overwrite existing load plans option is not selected.

Support for manual load plan execution

You can now manually execute a load plan from the Scheduler > Load Plans regardless of its schedule status: Active, Suspended, Completed, or Not Scheduled. Only users who own or have Edit access to all schemas in the load plan can execute it manually. Manually executing a load plan from the Load Plans list will not impact the next scheduled run if exists.

Default schema load type at the load plan level

You can now specify the default load type for schemas that you will add to a load plan. Whenever you add a new schema to the load plan, it inherits the default load type. If you change the default load type, it will not affect the schemas you have already added.

Load Job Details Viewer enhancements

• Displaying the name of the executed load plan if available.
• Displaying an indicator of the rejected rows per load group.
• Table details enhancements:
  • Display the deduplication phase for tables, which tracks the duration and status of the PK-index creation and Parquet compaction processes.
  • The ability to collapse and expand schema objects.
  • The ability to filter by the table load status and preserve the filter during navigation where applicable.
  • The ability to change the column width.

Support for merging Parquet segments during loading from staging

A new Spark job can run as part of load-from-staging jobs to merge Parquet segments, which are the result of incremental load jobs. This new step increases the system resilience and enhances the performance in clusters with an increased number of small Parquet files.

To enable this feature, turn on the Enable automatic merging of parquet while loading from staging option in the CMC Server Configurations > Tuning. However, during load from staging jobs, the following conditions must be met, along with enabling the feature, to start the merging process for an object:

• The Parquet Long to Int Migration option is turned off or the object doesn’t require migrating Long data to Integer.
• The object’s recent compacted version matches the recently extracted version, otherwise, the Loader Service performs a compaction recovery process before merging. If the recovery fails, the merge process will not start.
• The table has over 1000 eligible Parquet segments for merging.
• The estimated reduction in the number of files exceeds 50%.

Contact Incorta Support to change these configurations by editing the Loader Service’s engine.properties file and setting different values for the loader.parquet.merge.min.file.count and loader.parquet.merge.compression.ratio.threshold properties. Setting these properties to 0 or 1 means to merge whatever the file count or the gain percentage is.

Notes:

• The Parquet merge step creates a new Parquet version of the object on the disk (similar to the version created as a result of a full load job).
• The Parquet merge does not change the data merged. However, it changes the order of the records in the output Parquet files. After the merge, dashboards will show the same data, but in different order (if no sorting is applied).

Enhanced the responsiveness of Post-load interruption

This release enhances the responsiveness of interrupting load jobs during Post-load calculations. Previously, the Loader Service would wait for running calculations to complete before stopping the load job.

Pause Scheduled Jobs enhancements

Starting this release, you can choose which scheduled jobs you need to pause. Incorta replaced the “Pause Scheduled Jobs” toggle in the CMC with three toggles to give you more flexibility on which scheduler jobs you want to pause.

Support OAuth for JDBC connection

Incorta now supports OAuth for JDBC connections. You can enable this option from the CMC. Log into the CMC and Navigate to Clusters > cluster-name > Cluster Configurations > Default Tenant Configurations > Integration. Enable the option OAuth 2.0-based authentication for JDBC connection, knowing that by enabling this option you cannot use personal access tokens for JDBC authentication. Enter your authorization server URL and then restart the Analytics service.

Monitoring the file system usage

This release introduces a new feature to monitor how Incorta services use the file system and to collect the file system metrics. For now, this feature supports Google Cloud Storage (GCS) only.

This feature is disabled by default. Contact Incorta Support to enable the feature for each service that you want to monitor its requests (method calls) and set the interval to log the file system metrics in the service log file and the newly introduced file system audit file. Additionally, the feature can be configured to log detailed metrics in the tenant log file; however, it is not recommended to always enable this property as it might cause performance degradation.

Enhanced performance with parallel snapshot reading

Now, Incorta can read the snapshot DMM files of joins and formula columns in parallel using multiple threads to enhance and speed up reading these files. The performance enhancement may vary according to the number of columns and joins to read concurrently and the available resources.

This feature is disabled by default. To enable it, contact Incorta Support.

Advanced SQL Interface

In this release, Incorta is introducing an advanced version of the current SQLi that is fully Spark SQL compliant. With this compliance, the new SQLi introduces a new enhanced performance and compatibility with more external tools. You can also use the advanced SQLi with Incorta using the Kyuubi connector.

The Advanced SQL interface is disabled by default. To enable it, you must log into your Cloud Admin Portal (CAP) and turn on the Enable Advanced SQL Interface toggle the Configurations tab.

When enabled, the connection string is generated for you to use in your third-party tools or Incorta via the Kyuubi connector.

Limitations

• Only a single analytics service is supported.
• Geo data type is not supported.

Additional Considerations In case of using a tenant with the same name as a previously deleted one, please contact Incorta Support so you can avoid facing issues while using the Advanced SQL Interface. This issue has been resolved starting 2024.1.3 Maintenance Pack.

Fixes and Enhancements

In addition to the new features and major enhancements mentioned above, this release introduces some additional enhancements and fixes that help make Incorta more stable, engaging, and reliable.

Enhancements

Fixes

2024.1.1 Maintenance Pack

Incorta copilot for story telling

With this feature, Incorta enhances the capabilities of dashboard developers, enabling them to construct an entire dashboard using natural language. You can find a new Generate Story button in the Action bar within the Content Manager.

When you select the Generate Story button in Incorta, a dialog box will appear. This box will ask you to specify the business schema and business view that you want to use for your dashboard. Additionally, you need to provide a brief question or desired report outline to generate your dashboard.

Send a single insight

You can now schedule the sending of a single insight from a dashboard without the need to send the entire dashboard, utilizing the Send/Schedule a Report option. This feature is introduced to facilitate the sending of tabular insights that include pagination.

You can access this capability in the More Options menu of an insight, replacing the previous Send to Data Destination option. Upon choosing the Send/Schedule a Report option, Incorta opens the Scheduler, allowing you to configure the delivery of the insight.

Enhancements

2024.1.2 Maintenance Pack

Derived tables’ support for key columns

Now, you can specify key columns for Incorta Analyzer and SQL tables. Adding, removing, or changing key columns does not require running a load job, as derived tables are refreshed as part of schema update jobs. The derived table’s unique index is calculated and saved as a snapshot DDM file each time the key columns are updated or the schema or table is loaded.

Ensure that the column or columns you designate as key maintain row uniqueness because no deduplication is performed for derived tables. If the selected key columns result in duplicate key values:

• During the schema update job, duplicate values are kept, and the Engine will return the first matching value whenever a single value of the key columns is required. The schema update logs will point out the unique index issue.
• During the schema or table load job, the unique index calculation will fail, resulting in a finished-with-error load job. No value is returned when the unique index is required. You must select the correct key columns to have the unique index calculated.

Enhancements

2024.1.3 Maintenance Pack

What's new

• Relaxing the lookup function validation
• Multiple default groups by dimensions in aggregated table
• Dynamic Pivot Table Analysis
• Wrap the Label Text in the Dual X-axis chart
• Export Current Dashboard State
• Improved Handling of Cyclic Dependencies in Variables
• Querying non-optimized tables via Advanced SQL Interface

Relaxing the lookup function validation

The validation on the lookup() function introduced in 2024.1.0 is now relaxed to allow:

• Using columns from business schema views with columns from the source physical schema object in the same expression
• Limiting the validation to creating new or updating existing expressions
• Existing expressions where the result lookup field and primary key field parameters are not from the same object or its business schema views not to return an #Error value

Multiple default groups by dimensions in aggregated table

The dynamic group-by is a powerful feature in Incorta's aggregated tables. In the latest update, users can override the default grouping behavior in which the first dimension in the grouping dimension tray is displayed. Instead, users can select which grouping dimension(s) to show as a default for dashboard consumers.

Dynamic Pivot Table Analysis

In this release, pivot tables have added two powerful enhancements:

• The ability to define a dynamic group by logic (previously only available in aggregated tables)
• The ability to define dynamic columns

As a result, users can interactively select and de-select rows and columns to display in their pivot table. As a bonus, these two new features also allow users to set a default view.

Wrap the Label Text in the Dual X-axis chart

There is now a new lab format option to wrap the text label for the Dual X-axis chart. This format option applies to the upper X-axis. You can keep the original behavior, where the text exceeding the reserved area of the label is trimmed, or toggle label wrapping in the pill settings.

Export Current Dashboard State

Incorta can now export dashboards for PDF and HTML in their current view state in this release instead of reverting to their default state. The following actions supported from an end-user interaction that can be exported are:

• Sorting
• Dynamic Group By (in tables)
• Dynamic Measures (in tables)
• Dynamic Fields (used in measures)

If a dashboard consumer has used the above actions before exporting, the exported in a direct or scheduled dashboard export will reflect their changes. When exporting the dashboard, under the Choose Bookmark option, you can select existing bookmarks or “Current Dashboard State”. Note that dashboard states have the option to save to bookmarks.

Improved Handling of Cyclic Dependencies in Variables

This enhancement focuses on improving system resilience and stability by resolving issues related to infinite loops and overflow exceptions caused by cyclic dependencies in variables. However, it's essential to acknowledge that a standardized error message or expected output has not yet been established. Users may encounter varied outputs, including #Error, 0 rows, empty strings, or variable name displays, depending on the context. Ongoing efforts are dedicated to achieving uniform error handling and output consistency.

Querying non-optimized tables via Advanced SQL Interface

Non-optimized tables can now be accessible via the Advanced SQL Interface. As a result, external tools, such as Tableau and Power BI, can now discover and query non-optimized tables that do not have any of the following:

• Security filters
• Formula columns
• Encrypted columns

Contact Incorta Support to help you configure your cluster.

Enhancements

Known Issues

For all of the known issues and workarounds in Incorta Cloud releases, refer to Known Issues in 2024 Releases.