Tools → Schema Designer → Load Jobs

About the Schema Designer Load Jobs

You can use the Schema Designer to perform or schedule load jobs of all objects in the physical schema or to load a specific object.

The Summary section in the Schema Designer shows the status of the last load job or the stage of the current load job. You can review the details of the current or previous load jobs in the Load Job Viewer or the Load Job Details Viewer (starting with 2023.4.0).

Incorta supports different types and strategies for loading data. To learn more about loading data into Incorta, refer to References → Data Ingestion and Loading.

Changes that require data load

Some updates you make to the physical schema objects require loading data, either load from source (full load) or load from staging, to ensure data consistency.

The following are the updates that require load from staging:

  • Adding, updating, or removing the load filter
  • Enabling performance optimization for non-optimized objects

The following are the updates that require full load:

  • Adding a new physical schema table or materialized view
  • Changing the data type of a physical schema table column or MV column
  • Changing the source of a materialized view or physical schema table, whether by selecting another source file in the Data Source properties dialog or editing the query
  • Adding or changing a key column (changing the column function from key to dimension or measure and vice versa) in a physical schema table or MV
  • Adding a new physical schema table column
  • Adding a new MV column
  • Changing the object type, for example, changing a physical schema table to an Incorta Analyzer table or MV
  • Removing a physical schema table column or a materialized view column that functions as a key
  • Changing the encryption status of one or more columns in a physical schema table or materialized view
Warning

Incorta does not recommend running concurrent schema model update jobs and load jobs on the same schema or dependent schemas as this may result in errors or inaccurate data.


Schema Designer actions for load jobs

With the Schema Designer, you can perform the following actions for data load jobs

Change the load order for physical schema objects

  • In the Navigation bar, select Schema.
  • In the list view of schemas, select a given physical schema.
  • In the Schema Designer, select Settings (gear icon).
  • Select Change Load Order.
  • In the Choose Load Order dialog, you can do the following to manage the table groups before changing their load order:
    • For each table (or materialized view) that you want to add to a new load group, select the check box, and then select Add to New Group.
    • For each table (or materialized view) that you want to add to an existing load group, select the check box, and then select Add to Existing Group.
    • To reset table grouping and order, select Revert to Default.
  • To set the order of a table group, select the up arrow (to the right of the group name) to move the group upwards, or select the down arrow (to the right of the group name) to move the group downwards. Repeat this action until you order the table groups as required.
  • Select Save.

Perform a full load of the physical schema data

  • In the Schema Designer, in the Action bar, select LoadFull Load.
  • In the dialog, select Load.

Perform an incremental load of the physical schema data

  • In the Schema Designer, in the Action bar, select LoadIncremental Load.
  • In the dialog, select Load.

Load physical schema data from staging

  • In the Schema Designer, in the Action bar, select LoadStaging Load.
  • In the dialog, select Load.

Schedule a physical schema load job

  • In the Schema Designer, in the Action bar, select LoadSchedule Load Job.
  • In the Add Schema Load dialog, in Schema Load Name, enter the name of the scheduled load job. You can enter up to 255 alphanumeric characters, including spaces, special characters, and emojis.
  • In Description, optionally enter a description for the scheduled load job. You can enter up to 255 alphanumeric characters, including spaces, special characters, and emojis.
  • The Schema box shows the name of the current physical schema.
  • In Load Type, select the load type. The available options are:
    • Full
    • Incremental (default)
    • Staging
  • In the Recur Every section, specify the frequency and recurrences of this job.
    • Enter a number.
    • Select the time measure unit: minutes, hours, days, or months.
      • If you select Week, specify the days to run the job on.
      • If you select Month(s), select the day or weekday you want to run the job, for example, the second Monday of the month.
    • Select the No Recurrence check box to run this job only once.
  • In the At section, specify the time to run this job. Select the time and time zone.
  • Specify the validity period of the scheduled load job:
    • If the job is recurring, in the From section, do one of the following:
      • Select the start date and keep the Does not end check box selected to run it forever.
      • Select the start date, clear the Does not end check box, and then select the end date to run the job during this period.
    • If the job is not recurring, in the On section, select the date to run this job.
  • Select Add.
Note

The scheduled load jobs appear in the Scheduler, under Schema Loads. A user with the appropriate access rights to the Scheduler can manage these jobs, in addition to creating new jobs.

Perform a load of a physical schema object

  • In the Schema Designer, in the Tables tab, for a given physical schema object, select More Options (⋮ vertical ellipsis).
  • Do one of the following:
    • Select Load Table to start a full load job.
    • Select Load from Staging to start a load from staging job.
  • In the dialog, select Load.

Review the status of a load job

Create a physical schema load email notification

Important

Notifications require a tenant email configuration for an outgoing email server using SMTP or EWS in the Cluster Management Console (CMC).

The notifications that you create appear on the Scheduler, under Schema Notifications. A user with the appropriate access rights to the Scheduler can manage these notifications and create new notifications.

Here are the steps to create an email notification for a physical schema:

  • In the Navigation bar, select Schema.
  • In the list view of schemas, select a given physical schema.
  • In the Schema Designer, in the Action bar, select Settings (gear icon).
  • Select Create Email Notification.
  • In the Create Email Notification dialog, enter the details of the email notification. For more information, see Email notification details.
  • Select Done.

Email notification details

The following table shows the details you need to specify for a load email notification:

PropertyControlDescription
Notification Nametext boxEnter the name of the new email notification. The length of the notification name must be between 1 and 255 alphanumeric characters, including spaces and special characters.
Notify On → SuccesscheckboxSelect the checkbox to send the notification email when the load successfully completes
Notify On → FailurecheckboxSelect the checkbox to send the notification email when the load fails
Select Schema(s) → Searchtext boxEnter a search term to search for specific physical schemas to create load notification for.
Select Schema(s) → Select AllcheckboxSelect the checkbox to select all physical schemas you have access to or all physical schema(s) in the search result list
Select Schema(s) → Schemas listlist/checkboxSelect the checkbox for each physical schema you want to include in the load notification. Only physical schemas that you own or have access rights to appear on the list.
Recipients → Search Names, Emails, and Groupstext box
  ●  Enter a search term (for example, an email address, user name, group name) to search for users or groups in the tenant.
  ●  Enter the email address to select an external recipient.
Recipients → ListlistFor each recipient, select V (down arrow) to the right of the recipient name or email, and then select where the given recipient should go to: the To list, Cc list, or Bcc list.
Bodytext boxOptionally, enter the email body text. The body text you enter appears in the notification email before an auto-generated text. You can enter up to 4000 alphanumeric characters, including spaces and special characters.
Note

When you create an email notification that contains multiple physical schemas, the recipient(s) receives a separate email for each physical schema. Recipients on the Bcc list do not appear on the received email, while those on the To and Cc lists do appear.

The notification email contains the following:

  • the email body text you entered, if any
  • the notification name
  • the display name of the notification user creator
  • the schema name
  • the schema load status
  • the load date and time
  • the load duration
  • a link to the Load Job Viewer that contains the schema load job summary and details