Guides → Configure Tenants
You can manage your tenant(s) through the Cluster Management Console (CMC) by configuring various settings, such as a minimum password length, default language, maximum cached memory, and other settings. Here are the steps to configure your tenant(s):
- Sign in to the CMC.
- Configure your tenants as follows:
- To set the configurations of a specific tenant, select Clusters > Cluster-name > Tenants, and then for the tenant row, select Configure.
- To set your default tenant configurations, select Cluster Configurations > Default Tenant Configurations.
The following tenant configurations are available in the CMC:
- Security
- Regional Settings
- Data Loading
- Data Management
- Integration
- Export
- Advanced
- Tuning
- Incorta Labs
- External Visualization Tools
- MV Assistant
Warning: Save Changes
To avoid losing data after configuring your settings on a page, select Save before navigating to another page.
Important: Cloud Admin Limited Access
Some configurations are not available when using the cloud admin account to sign in to the CMC. These configurations are marked on this document with an asterisk (*). Contact Incorta Support to have any of these configurations set or altered.
Security
The security configurations include the authentication types and other important settings within Incorta Analytics, such as enabling Super User mode and support for password-protected Excel files.
The following table illustrates the security configuration properties:
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| Enable Inviting users | toggle | No | Enable this property to invite users by email. The Super User tenant admins and users with a SuperRole access right can invite external users to join their tenants. Note the following:
|
| Authentication Type | drop down list | No | Select the authentication type you support from the drop down list. The available types are: |
| Allow Different Authentication Type for External Tools | toggle | Yes, all services | Enable this property to authorize a different authentication type for external tools, and then select the authentication type from the External Tools Authentication Type drop down list. The supported types are:
|
| Enable Super User Mode | toggle | No | Enable this property to enable non-Admin users with the SuperRole to access all entities within a tenant. Disable this property to restrict access for all non-Admin users and provide access to shared entities only. Note: This option does not affect the access to folders and dashboards. Any user, including the Super User and users with the SuperRole must own or have access rights to a dashboard or folder to view it in the Content Manager. |
| Enable Support for Password-Protected Excel Files | toggle | No | Enable this property to support the upload of password-protected Excel files using the Data Manager. Prerequisite: you must install Java Cryptography Extension (JCE) to use this feature. Here are the steps to install JCE: ● Download and extract JCE from http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html ● Copy the JCE jars ( local\_policy.jar and US\_export\_policy.jar) into $JAVA\_HOME/jre/lib/security Restart Incorta CMC. |
| Download insights | toggle | No | Disable this toggle to limit users with User or Individual Analyzer user roles from downloading insights. |
Incorta Authentication
The following table illustrates the configuration properties of the Incorta authentication.
Warning
The default password policy has changed starting 2022.10.0. The default policy will be applied only to newly created passwords, whether for new users creating their first passwords or existing users changing their passwords. However, this will not require that existing users should change their passwords. The new password policy mandates the following:
- The minimum password length is 12 alphanumeric characters.
- A password must contain at least one uppercase letter, one lowercase letter, one special character, and one digit.
- A password cannot include the user login name or email address.
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| Authentication Type | drop down list | No | Select Incorta Authentication to sign in to Incorta using the Incorta Analytics username and password. You can also configure password rules and other security-related options. |
| Minimum Password Length | spin box | No | Enter the minimum number of characters for a password. The Default is 5 characters. |
| Password Cannot Include Username | toggle | No | Enable this property to restrict use of a username in passwords. |
| Require Lower Case Letters | toggle | No | Enable this property to require lower-case letters in passwords. |
| Require Upper Case Letters | toggle | No | Enable this property to require upper-case letters in passwords. |
| Require Digits | toggle | No | Enable this property to require numbers in passwords. |
| Require Special Characters | toggle | No | Enable this property to require special characters in passwords. |
| Enable Password Expiration | toggle | No | Turn this toggle on to force Incorta authenticated user to change their password at a scheduled cadence. |
| Password validity in days | text box | No | Enter the password validity period in days. This property is available when you turn the Enable Password Expiration toggle on. The valid value is between 1 and 365 inclusive. The default value is 90 days. |
Note
The password expiration feature is available starting the 2022.10.0 release and it is disabled by default.
- The start of the password validity period is the password modification date.
- After upgrading to 2022.10.0, the password modification date will be the upgrade date.
- For newly created user accounts, the password modification date will be the day when the user creates a new password after successfully signing in to Incorta for the first time.
- When a password expires and the user tries to sign into Incorta, a message appears denoting that the password has expired. The user has to enter the current password and create a new one.
- A User Manager or a Super User can also use the
resetPasswordspublic API endpoint to instantly reset the password of all Incorta authenticated users.
The password policy options related to Incorta authentication will remain visible even when you select another authentication type. However, they will affect Incorta authenticated users only, such as the Super User and other users you specify Incorta as their authentication method.
Single Sign-On
The following table illustrates the configuration properties of the Single Sign-On (SSO) authentication:
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| Authentication Type | drop down list | Yes, if there are changes in the 'server.xml' file | Select SSO to sign in to Incorta using a standard SSO tool. This requires enabling Allow Different Authentication Type for External Tools. For more information, refer to Guides → Configure SSO. |
| Provider type | drop down list | No | Select your provider type. The options are: ● SAML2 ● Okta ● Auth0 ● Custom (requires entering the name of the custom provider for a specific customer) |
| Provider configurations | text box | No | Enter the properties or XML configurations for your selected provider. |
| Auto provision SSO users | toggle | No | Enable this property to automatically add all SSO users to Incorta after successful authentication. |
| Auto provisioned SSO users default group | drop down list | No | Select the newly provisioned SSO users to be assigned to this group. |
Lightweight Directory Access Protocol
The following table illustrates the configuration properties of the Lightweight Directory Access Protocol (LDAP) authentication:
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| Authentication Type | drop down list | No | Select LDAP to sign in to Incorta using the LDAP system user and password. |
| Base Provider URL | text box | Yes, all services | Enter a URL to access the LDAP server in the following format: ldap://<LDAP_SERVER_HOST>:<LDAP_SERVER_PORT> |
| Base Distinguished Name | text box | Yes, all services | Enter a name attribute to describe the location where the LDAP search should begin. |
| System User | text box | Yes, all services | Enter the LDAP System User attribute. If searching your LDAP server does not require authentication, do not set this property. |
| System User Password | text box | Yes, all services | Enter the password for the LDAP System User. |
| User Mapping Login | text box | Yes, all services | Enter the attribute in LDAP that can be used to sign in to Incorta Analytics. This is the name of the column that contains a unique identifier for users in LDAP, such as an email or ID. |
| User Mapping Authentication | text box | Yes, all services | Optionally, enter an attribute to authenticate users. If no attribute is entered, the value entered for User Mapping Login will be used to authenticate users. |
Azure Active Directory
The following table illustrates the configuration properties of the Azure Active Directory (AD) authentication:
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| Authentication Type | drop down list | No | Select Azure AD to sign in to Incorta using the Client ID and Client Secret. For more information on how to obtain the Azure AD Client ID and Client Secret, refer to Quickstart: Register an app in the Microsoft identity platform. |
| Client ID | text box | No | Enter the Application (Client) ID for the App registration defined on the Azure AD. |
| Client Secret | text box | No | Enter the Client Secret value (application password) that is defined for the Client ID on the Azure AD. |
Regional Settings
The following table illustrates the configuration properties of the regional settings:
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| Language | drop down list | No | Select the default language for your tenant using the ISO two-letter codes. For example, fr stands for the French language. |
| Country | drop down list | No | Select the default country for your tenant. |
| Text Direction | drop down list | No | Select the text direction for your tenant:
|
The following table illustrates the configuration properties of the tenant email:
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| SMTP Host | text box | No | Enter the server used to send emails. For example, smtp.gmail.com. |
| Enable SMTP SSL | toggle | No | Enable this property to establish a port for Simple Mail Transfer Protocol Secure (SMTP) to operate using SSL. |
| Socket Factory Port | spin box | No | Enter the port number used for connection when using an SMTP Socket Factory. |
| Email Host Requires Authentication | toggle | No | Enable this property if the email host, such as Gmail, requires authentication before sending an email. |
| SMTP Server Port | spin box | No | Enter the port number to use for connecting to the email server. |
| Email Server Protocol | drop down list | No | Select an email protocol:
|
| Export Server Timeout in Seconds* | spin box | No | Enter the required server timeout in seconds when sending an HTML dashboard. If a dashboard is not sent due to a server timeout, you can increase this value to resolve the issue. |
| Sender's Username Auth | toggle | No | Enable this property to use a username for the SMTP authentication instead of the sender's email. |
| System Email Username | text box | No | Enter the SMTP username. This property is available when Sender's Username Auth is enabled. |
| System Email Address | text box | No | Enter the email address to use for sending dashboards or notifications. |
| Person Name* | text box | No | Enter the name to display as the email sender. |
| System Email Password | text box | No | Optionally, enter the account password or an app password for the email account used for sending dashboards or notifications. Leave this field blank if the email server does not require a password for the sender’s email. |
| Forgot Password Email Subject* | text box | No | Enter the text to display in the subject line of password recovery emails. |
| Server URL Protocol* | drop down list | No | Select the server url protocol:
|
| Server Name* | text box | No | Enter the current server name, for example, localhost. |
| Server Port* | spin box | No | Enter the current server port, for example, 8080. |
| Enable Sharing Notifications | toggle | No | Enable this property to send email notifications when sharing dashboards. |
| Notification Email Subject* | text box | No | Enter the subject line of a sharing notification email. |
| Enable Internal Error Notifications | toggle | No | Enable this property to be notified of the internal error(s) by email. |
| Internal Error Notification Subject* | text box | No | Enter the subject line of the internal error notification emails. |
Note
If you have set up two-step verification for the email account you use to send scheduled dashboards and notifications, you must use an app password instead of the account password for the System Email Password option.
For details about how to create an app password, refer to the documentation of your email service provider. For example, for an Office 365 email account, refer to Manage app passwords for two-step verification, and for a Gmail account, refer to Sign in with app passwords.
Data Loading
The following table illustrates the configuration properties related to data loading:
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| Enable Always Compact | toggle | No | In releases before 2023.1.0, enable this property to trigger automatic compaction during schema load. Note the following:
|
| Stop Loading on First Error | toggle | No | Enable this property to stop a schema extraction or loading on the first encountered error. |
| Pause Scheduled Jobs | toggle | No | Enable this property to pause all the active schedule jobs including schema loads, dashboards, and data alerts. |
Data Management
The following table illustrates the configuration properties related to data management:
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| Stop Loading on First Error | toggle | No | Enable this property to stop a schema extraction or loading on the first encountered error. |
| Pause Load Plans | toggle | No | Enable this property to pause all the active load plans. |
| Pause Dashboard Scheduler | toggle | No | Enable this property to pause all the active scheduled dashboards. |
| Pause Data Notifications | toggle | No | Enable this property to pause all the active data alerts. |
| Earliest Valid Year | spin box | No | Enter the earliest year you want to extract from your data sources. Years prior to the entered date will be extracted as NULL. Default value is 0, that will allow all years to be extracted. |
Integration
The following table illustrates the integration configuration properties:
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| Google Maps API Key | text box | No | Enter the API key for Google Maps. |
| Apple Maps Developer Team ID | text box | No | Enter the Developer team ID to use for enabling Apple Maps in Incorta’s maps visualizations. |
| Apple MapKit JS Key ID | text box | No | Enter the Apple MapKit JS key to use for enabling Apple Maps in Incorta’s maps visualizations. |
| Apple Maps API Key | text box | No | Enter the Apple Maps API key to use for enabling Apple Maps in Incorta’s maps visualizations. |
| Google Drive Client ID | text box | Yes, all services | Enter the Google Drive Client ID used by the Google Drive connector. |
| Google Drive Client Secret | text box | Yes, all services | Enter the Google Drive Client Secret used by the Google Drive connector. |
| Dropbox App Key | text box | Yes, all services | Enter the Dropbox App Key used by the Dropbox connector. |
| Dropbox App Secret | text box | Yes, all services | Enter the Dropbox App Secret used by the Dropbox connector. |
| Box Client ID | text box | Yes, all services | Enter the Box Client ID obtained from the Box application. |
| Box Client Secret | text box | Yes, all services | Enter the Box Client Secret obtained from the Box application. |
| Mapbox API Key | text box | No | Enter the Mapbox API Key of the Advanced Mapbox. |
| Incorta Slack Integration | toggle | No | Available starting the 2024.1.0 release. Enable to allow the integration between Incorta and your Slack workspace. |
| Slack app Client ID | text box | No | Enter your unique Slack app client identification number. |
| Slack app Client Secret | text box | No | Enter your application client secret number found in your App Credentials section in Slack. |
| Incorta Slack bot token | text box | No | Enter the bot user OAuth token for your Slack workspace. |
| Incorta Microsoft Teams Integration | toggle | No | Available starting the 2024.1.0 release. Enable to allow the integration between Incorta and your Microsoft Teams. |
| OAuth 2.0-based authentication for JDBC connection | toggle | No | Available starting the 2024.1.0 release. Enable this feature to allow OAuth 2.0 client applications to authenticate with Incorta JDBC. Note that when this feature is enabled, the use of personal access tokens for JDBC authentication will no longer be possible. |
Export
The following table illustrates the configuration properties related to exporting Insights:
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| Export All Data from Listing or Aggregated Tables* (Before 2024.1.0: Export All Data) | toggle | No | Enable this property to export or download all data from Listing or Aggregated tables to CSV or Excel files. Enabling this option will override the row limit set in the Download Rows Limit option. |
| Export All Data from Pivot Tables* (Before 2024.1.0: Export All Data (Pivot Tables)) | toggle | No | Enable this property to export or download all data from Pivot tables to Excel files. Enabling this option will override the row limit set in the Download Rows Limit option. |
| Download Rows Limit* | spin box | No | Enter the maximum number of rows to download or export data from a Listing, Aggregated, or Pivot table to a CSV or Excel file. This option applies only to exporting from Listing or Aggregated tables to CSV or Excel files when Export All Data from Listing or Aggregated Tables is disabled. Similarly, this option is effective only for exporting from Pivot tables to Excel files when Export All Data from Pivot Tables is disabled. |
| Add BOM to CSV* | toggle | No | Enable this property to add a Byte Order Mark (BOM) to CSV files. This property must be enabled to export Unicode-encoded files. |
| Include Prompt Selections in Excel Export* | toggle | No | Enable this property to add the applied prompt filters to the first line in an exported or downloaded Excel file. |
| Export Folder* | upload | No | Select + Add Folder to enter the destination folder’s name and path on the Incorta host server to use for sending or scheduling a dashboard. |
Advanced
The following table illustrates the advanced configuration properties related to materialized views (MVs) and the maximum groups default and limit in insights.
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| Enable Hive Context in MVs* | toggle | No | Enable this property to support Hive context in MVs. |
| Python Path | text box | No | Enter the Python binary executable to use for Python MVs in the driver and executors. |
| Insight Max Groups UI Default | spin box | No | Enter the default maximum number of groups that cannot be exceeded in an insight. You can also configure this property for an insight in the Settings panel of the Analyzer. The value in the insight overrides the value configured in the CMC. |
| Insight Max Groups Limit | spin box | No | Enter the limit for the maximum number of groups that cannot be exceeded in an insight. Entering the values 0 or -1 will set the number of groups to unlimited. This value cannot be overridden. |
| Insight Max Values in Filter Subquery | spin box | No | Enter the maximum values for an insight in the filter subquery. |
| Force Reload Columns | toggle | Yes, Analytics Service only | Enable this property to automatically reload the data in memory after a schema load. For example, after an incremental update. If disabled, data will only be reloaded when accessing a dashboard that queries such data, which may impact the performance and cause a delayed response. |
| Sync in Background* | toggle | Yes, Analytics Service only | Enable this property to synchronize your data in the background during a schema load by reading columns in separate memory on the Analytics service. |
| Warmup Mode on Analytics Services | drop down list | No | Select an option to load and warmup specific columns first on the Analytics service. This decreases the time to load dashboard queries. The options are: ● None - works best for small deployments with ad-hoc queries. ● Last Used Columns - load the previous state prior to shut down in the analytics service. ● Business View Columns - load all columns referenced in business schema views on the Analytics service only. ● Most Used Dashboards Columns - load the most used dashboards columns first. ● All - works best for supporting ad-hoc queries, in case no business schemas are in place, and the time between the Analytics service startup and dashboard usage is significant. |
| Warmup Mode on Loader Services | drop down list | No | Select an option to load and warmup specific columns first on the Loader service. The options are: ● None - works best for small deployments with ad-hoc queries. ● Last Used Columns - load the previous state prior to shut down in the Loader service. ● All - works best for supporting ad-hoc queries. |
| Turn off/on Global Auto Refresh for Insights | toggle | No | Enable this property to turn on Auto Refresh by default for all insights. You can turn off Auto Refresh to allow the Analyzer Users to turn on Auto Refresh for individual insights. |
Tuning
The following table illustrates the tuning configuration properties:
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| Maximum Concurrent Load Jobs | drop down list | Yes, Loader Service only | Select the maximum number of load jobs (up to 10) that the Loader Service can handle at the same time, regardless of the number of physical schemas in each load job. The default value is 5. This property is available starting the 2022.11.0 release. |
| Schema Pool Size | drop down list | Yes, Loader Service only | Select the number of schemas (up to 10) that can be loaded at a time per tenant. The default value is 5. This option is available in releases before 2022.11.0. |
| Table Maximum Parallel Chunks* | drop down list | Yes | Select the number of concurrent chunks (between -1 and 10) that you can extract from the source at the same time. |
| Individual and dashboard filter optimization | toggle | No | Enable this property to improve the performance of filtering data, and accordingly reduce the overall rendering time of insights and dashboards. This feature optimizes filtering data using insight individual filters and when applying runtime security filters and dashboard filters (including dashboard runtime filters, applied filters, dashboard prompts, and filter options). This property doesn’t affect queries with non-regular expressions, such as First Version, Last Version, and Distinct. |
| Maximum Cached Entries | spin box | Yes, Analytics Service only | Enter the maximum number of insights or MVs to store in the cache. The default value is 2000. Caching an insight larger than 10MB may affect performance. Incorta uses both the Maximum Cached Entries and the Maximum Cached Memory (%) properties to determine the cache size, and then uses the lower of the two limits. For example, if you set off-heap memory to 100GB, and Maximum Cached Memory (%) to 1%, the cache size will be 1GB. Therefore, if you set Maximum Cached Entries to 2000 and you reach 1GB with fewer reports, Incorta will stop caching reports at 1GB. |
| Maximum Cached Memory (%) | spin box | Yes, Analytics Service only | Enter a percentage between 0% and 10% of the total off-heap memory to use as the maximum memory size for cached reports. The default is 5% of the off-heap memory. |
| Export to CSV/XLSX caching limit (In megabytes) | spin box | Yes, Analytics Service only | The maximum data size of the query result that the Analytics Service caches in memory for insights downloaded, sent to data destinations, sent to a download folder, or shared via email in XLSX and CSV formats. The Analytics Service checks for a cached version before executing the insight query, regardless of the requesting user. Valid values are between 0 and 100 inclusive. The Analytics Service won’t cache insights in the following cases: ● You set this property to 0. ● The query result size exceeds the limit you define. ● You set the Maximum Cached Memory (%) or Maximum Cached Entries options to 0, or these options reach their maximum limits. ● You download or share a dashboard or tab with multiple insights in XLSX format. Note: When you download or share a dashboard or tab with multiple insights in CSV format, only the first supported insight is downloaded or shared, and cached if applicable. |
| Evaluate Session Variables At Login* | toggle | Yes | Enable this property to evaluate all session variables on a separate thread at session login. This allows you to evaluate several session variables used in different insights on a single dashboard. |
| Query timeout | toggle | No | Enable this property to allow queries to be terminated automatically if they exceed the specified limit to run, releasing the read lock acquired on the related entities and freeing the resources reserved by these queries. This property affects querying data in the following scenarios: ● Creating an insight ● Rendering a dashboard ● Sending a dashboard via email or to a data destination ● Downloading a dashboard as CSV or Excel ● Previewing a chart as a listing or aggregated table ● Creating or updating an Incorta Analyzer table or view ● Exploring the data of a table or view ● Creating a data notification condition |
| Query timeout (in minutes) | spin box | No | Specify a time limit after which running queries will be terminated automatically. The default is 10 minutes. This property is available only when Query timeout is enabled. |
Note
The Individual and dashboard filter optimization and Query timeout features are available starting with the 2023.4.0 release while the Export to CSV/XLSX caching limit (In megabytes) feature is available starting with 2023.4.1. Starting with the 2023.7.0, The Individual and dashboard filter optimization option is available to the cloud admin user.
Incorta Labs
Note
An Incorta Labs feature is experimental and can be either promoted to a product feature or deprecated without notice.
The following table illustrates the configuration properties of Incorta Labs:
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| Start Engine on a background thread* | toggle | No | Enable this property to start the engine on a background thread. |
| Enable saving query plan | toggle | No | Enable this property to keep the join paths of your insights fixed. If a change in a data model causes an invalid join path, the affected insights will remain the same unless their owners approve to update them. |
| Incorta-Assist | toggle | No | Enable this property to have a bot assist you by suggesting and creating joins in a matter of minutes. |
| Enable Insight 'View As' Menu | toggle | No | Enable this property to show the View As menu in all the Dashboard Insight action menus. |
| Enable Incorta SQL Table | toggle | No | Enable this property to create an Incorta SQL Table. The Incorta SQL Table will use a new SQL Engine that supports all types of joins and provides better SQL coverage. The new SQL engine will query from data loaded in the memory. |
| Machine Learning support* | toggle | No | Enable this property to have machine learning support. |
| Enable Visualizations SDK* | toggle | No | Enable this property to use external visualizations from the marketplace. For more information, refer to Incorta Component SDK. |
| Enable creation of SQL Views | toggle | No | Enable this property to create Incorta SQL View using SQL. SQL in the Incorta SQL View uses a new SQL Engine that supports all types of joins and provides better SQL coverage. Dashboards can use SQL View to query and analyze data. The output of SQL View is generated during run time. |
| Enable internal join cache for SQL Engine* | toggle | No | Enable this property to internally join the cache for the SQL Engine. This will improve the SQL performance of SQL View. |
| Enable Insights over Insights* | toggle | No | Enable this property to create insights over result sets |
| Enable Bursting Reports* | toggle | No | Enable this setting to allow Incorta to distribute reports based on the recipients’ context instead of the sender’s context. You can send or schedule a bursting report or dashboard for up to 300 internal users with Incorta accounts and at least view access rights to the dashboard. The report runs once but renders customized data based on the security filters or session variables of each recipient. For more details, refer to Bursting Reports. |
| Max concurrent chromiums* | text box | Yes, Analytics Service only | Specify the max number of concurrent headless chromium browsers |
| Max no. of selections for contains filter | spin box | No | Default is -1, which indicates unlimited number of selections. Enter the maximum number of selected values in dashboard prompts for string mathcing operators (In, Not In, Starts With, Does Not Start With, Contains, Does Not Contain, Ends With). Note: 0 indicates unlimited numbers of selections as well. |
| Auto Detect Column Type | toggle | No | Disable this property to turn off auto-detection and columns update in a schema table with two or more data sources. |
| Maximum number of versions per entity | spin box | No | Enter the maximum number of versions to maintain for each entity, such as a schema or dashboard. The default value is 50. |
| Backup frequency | drop down list | No | Select the cadence to create versions for each entity. This can range from a record of each change to a weekly backup. The options are:
|
| Notebook Integration | toggle | No | Enable this property to integrate the notebook. Notebook is not currently supported on Windows environments. |
| Enable Dynamic Allocation in Notebook | toggle | No | Enable this property to use dynamic allocation by default in all notebooks. Disabling this property will cause notebook sessions to allocate the full resource of the materialized views in their interactive session. Thus, it is recommended to enable this property to avoid consuming Spark cluster resources by the notebook sessions. |
| Enable Multi-Schema Loading* | toggle | No | Turn this toggle on to be able to schedule load plans that involve loading multiple physical schemas. This option is available starting with 2022.11.0 and is deprecated in 2023.7.0 where the multi-schema load plans are general availability (GA) feature that is always enabled. |
External Visualization Tools
The following table illustrates the configuration properties related to the external visualization business tools:
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| Default Schemas* | text box | No | Enter the list of default schemas, separated by commas, to be processed in order when a non-fully qualified table is encountered in an SQL query from an external BI tool. |
| Incorta Host* | text box | No | Enter the IP address of the Incorta Server (or Proxy or Load Balancer, if applicable) to use for connecting the external BI tools to Incorta. |
| Incorta Port* | spin box | No | Enter the port number of the Incorta Server (or Proxy or Load Balancer, if applicable) to use for connecting the external BI tools to Incorta. |
| Excel Addin* | toggle | No | Enable this property to see instructions on how to connect Excel as an external BI tool to Incorta. |
| MicroStrategy* | toggle | No | Enable this property to see instructions on how to connect MicroStrategy as an external BI tool to Incorta. |
| Tableau* | toggle | No | Enable this property to see instructions on how to connect Tableau as an external BI tool to Incorta. |
| PowerBI* | toggle | No | Enable this property to see instructions on how to connect PowerBI as an external BI tool to Incorta. |
MV Assistant
Specifying the Spark-related configurations of each MV is a very important step toward improving the MV loading time and performance. However, it is not a simple task to identify the appropriate value combination of these configurations. The 2022.9.0 release introduces the MV Assistant feature to facilitate identifying the appropriate values for each MV. When the feature is enabled, Incorta collects various metrics during each MV load job, analyzes these metrics, and simulates different values to identify and recommend the optimal value combination.
Note
The MV Assistant is a preview feature in the 2022.9.0 release.
The MV Assistant mainly consists of a Spark Listener, a Heuristics Recommender, and a Cleanup job.
- The Spark Listener collects the required metrics while running the Spark application of an MV load job, simulates different values of the Spark configurations, and saves the output to a file in the following path:
<TENANT_NAME>/mvlenslogs/pending/<SCHEMA_NAME>/<TABLE_NAME>/. - The Heuristics Recommender uses the output file created by the Spark Listener and applies several heuristics to find the most optimal set of values and saves them to the metadata database. After using the output file, it is moved to the
<TENANT_NAME>/mvlenslogs/archived/<SCHEMA_NAME>/<TABLE_NAME>/. directory. - The Cleanup Job, when enabled, deletes the archived files that the Heuristics Recommender has already used after a specific number of days or after reaching the maximum number of archived files to keep. For details, see the MV Assistant Properties
The following are the configurations that the MV Assistant analyzes and recommends new values for.
- Executor instances (
spark.executor.instances): The total number of executors to allocate for the application - Executor cores (
spark.executor.cores): The number of cores per each executor - Executor memory (
spark.executor.memory): The amount of memory to be allocated for each executor - Shuffle partitions (
spark.sql.shuffle.partitions): Determines how many partitions should the data be partitioned into after shuffling
Recommended values are saved to the metadata database per MV and are introduced via a dedicated dashboard. Schema developers can manually apply the recommended values per MV.
Note
Contact Incorta Support to get the required assets.
Feature limitations
- The MV Assistant runs on the level of a single MV. Only one instance of the MV Assistant can run per tenant at a time.
- Recommended values cannot be automatically populated to the related MV or other MVs. You have to set them manually.
- This feature aims at enhancing the MV loading time and performance regardless of the cost and resources; however, they will be limited to the defined configurations.
- The MV Assistant does not consider the cluster size.
- The MV Assistant does not currently support the dynamic resource allocation option at the MV level or system-wide and is intended for those customers managing MV resource allocation with default configuration with overrides at the individual level.
- The recommended values are for both full and incremental load jobs.
Notes
- The MV Assistant requires at least 1 successful load for an MV to provide recommendations for it.
- The MV Assistant introduces recommendations when there is a chance of improvement in the loading time and performance, otherwise it will introduce no recommendations.
- Only MVs with recommendations are recorded in the database and represented in the dashboard.
- The improvement percentage and value are approximate.
- The MV Assistant consumes a very minor amount of system resources during each MV job submission.
MV Assistant Properties
The following table illustrates the available properties of the MV Assistant.
| Configuration Property | Control | Requires Restart | Description |
|---|---|---|---|
| MV Assistant | toggle | Yes, Loader Service only | Enable or disable the MV Assistant |
| Maximum amount of memory per core | spin box | Yes, Loader Service only | The maximum amount of memory in GB per each core, which will derive the maximum amount of memory that can be assigned to each executor. The accepted value is between 1 and 20. |
| Maximum number of total cores | spin box | Yes, Loader Service only | The maximum total number of cores the MV Assistant can recommend per Spark application. The accepted value is between 5 and 1000 |
| MV Assistant Interval (minutes) | spin box | Yes, Loader Service only | Specify the time interval to run the recommender component of the MV Assistant |
| Enable runs cleanup | toggle | Yes, Loader Service only | Enable or disable the cleaner job. The cleaner job deletes archived files created by the MV Assistant |
| Cleanup interval (days) | spin box | Yes, Loader Service only | Specify the time interval to run the cleaner |
| Retention period for runs (days) | spin box | Yes, Loader Service only | Specify the number of days to keep the files created by the MV Assistant during an archived run. This option, along with the Maximum number of archived runs per MV option, determines the files to keep and the ones to delete. Whatever option is satisfied first. |
| Maximum number of archived runs per MV | spin box | Yes, Loader Service only | Specify the maximum number of archived run files to keep. This option, along with the Retention period for runs (days) option, determines the files to keep and the ones to delete. Whatever option is satisfied first. |