Guides → Upgrade to the 2024.1.3 On-Premises Release

Prior to upgrade, please review the Release Model. The Release Model describes the various release statuses: Preview and Generally Available. Refer to the Release Model to identify the release version and status that is most suitable for upgrading your Sandbox, Developer, User Acceptance Testing (UAT), or Production environment.

Upgrade Considerations

Next-generation loader

With the introduction of the new-generation loader, Incorta automatically detects inter-object dependencies within a load plan during the Planning phase of a load job. The Loader Service utilizes these detected dependencies and the user-defined load order within the schema to create an execution plan for loading objects. However, it’s important to note that using both the MVs' user-defined load order and automatically detected dependencies may result in an execution plan with cyclic dependencies, leading to load job failures. To avoid such failures, it is recommended to delete the MVs' user-defined load order before upgrading from a release before 6.0.

Connectors Marketplace

Starting 2024.1.3 On-Premises release, Incorta has introduced the Connectors Marketplace, allowing administrators to install, upgrade, and downgrade connectors independently from any Incorta release.

After upgrading to 2024.1.3, the connectors’ files must be placed under the /marketplace/connectors/ directory. If this directory does not exist, create it under the system tenant path, which may vary according to Incorta installation. Afterward, you must unzip the ext_connectors.zip file that exists in the Incorta installation package and copy all the connectors’ folders from the unzipped directory to /marketplace/connectors/.
Incorta moved the custom CData connectors from <IncortaNode>/extensions/connectors/customCData/ to <IncortaNode>/extensions/connectors/shared-libs/.
For custom SQL connectors, you must move your files from <IncortaNode>/runtime/lib/ to <IncortaNode>/extensions/connectors/shared-libs/.
In case you need additional jars to use with the Oracle connector to support advanced options like XML, you must add or move the necessary libraries from <IncortaNode>/runtime/lib/ to <IncortaNode>/extensions/connectors/shared-libs/sql-oracle/.

Metadata database check

Upgrading to this release requires upgrading the metadata database to support multi-group load plans and migrate existing schema load jobs or load plans. Before upgrading to 2024.1.3, contact Incorta Support to perform a database check to inspect and detect any issues with your metadata database that might cause the metadata database upgrade to fail. If issues are detected, Incorta Support will run scripts against the metadata database to delete the records causing these issues.

Note

If the records that should be deleted are related to a multi-schema scheduled load job, the scheduled load job will be deleted altogether, and you will need to recreate the load plan manually if required.

SAP connector JCo files

Starting 2024.1.3, the SAP ERP Connector will stop bundling the JCo 3 libraries. For more information, refer to License Terms for SAP Connectors. Accordingly, you must manually get the needed libraries from SAP Java Connector and place them under <INCORTA_HOME>/IncortaNode/extensions/connectors/shared-libs/sapjco3.

Public API v2

The response of the /schema/{schemaName}/list endpoint has been updated. Some parameters have been renamed while additional parameters are now available to display error messages, the source column of runtime business view columns, and their data types. For more details, refer to Public API v2 → List Schema Objects.

Columns with Integer data type

In previous releases, Incorta wrote Integer columns in Parquet as Long. Starting 2024.1.3, Incorta writes Integer columns in Parquet as Integer for all newly created tables. For previously created tables, Incorta converts Integer columns written as Long in Parquet files to Integer during full load jobs, while Incorta keeps these columns unchanged during incremental loads.

As a result, after performing a full load of a table with Integer columns, it is recommended that you perform a full load of its dependent schema objects to ensure data consistency.

To migrate all Parquet files to have Integer data without fully loading your objects, the administrator can turn on the Enable Parquet Migration at Staging option in the Cluster Management Console (CMC) > Server Configurations > Tuning and perform a load from staging for all your objects with Integer columns.

Notes:

Turning the Enable Parquet Migration at Staging option on adds a new step that runs on Spark to the staging load jobs. Ensure Spark has sufficient resources to migrate the required Parquet files.
The migration of Parquet files during staging load occurs only once per object.
Load tables before loading MVs that read from these tables.
When loading an object fully, you must load dependent objects or load all objects from staging after turning the Enable Parquet Migration at Staging option on.
The new behavior might affect existing scripts referencing Integer or Long data, whether columns, values, or variables.

Schema names

Starting 2024.1.3, the names of newly created schemas and business schemas are not case-sensitive. Therefore, you cannot create a new schema or business schema that shares the same name with existing schemas or business schemas using different letter cases. To maintain backward compatibility, the names of existing schemas and business schemas will not be affected.

Caching mechanism enhancements

The caching mechanism in Analyzer views for dashboards has been enhanced by caching all columns in the view to prevent query inconsistencies. To optimize performance and reduce off-heap memory usage, creating views with only the essential columns used in your dashboards is recommended.

Like function fix

As a result of fixing an issue with the like() function, you must load all tables that use this function from staging if you are upgrading from releases before 6.0.2.