Tools → Schema Designer → Load Jobs
About the Schema Designer Load Jobs
You can use the Schema Designer to perform load jobs of all objects in the physical schema or to load a specific object. You can also create load plans to schedule loading one or more physical schemas.
The Summary section in the Schema Designer shows the status of the last load job or the stage of the current load job. You can review the details of the current or previous load jobs in the Load Job Details Viewer.
Incorta supports different types and strategies for loading data.
Changes that require data load
Some updates you make to the physical schema objects require loading data, either load from source (full load) or load from staging, to ensure data consistency.
The following are the updates that require load from staging:
- Adding, updating, or removing the load filter
- Enabling performance optimization for non-optimized objects
The following are the updates that require full load:
- Adding a new physical schema table or materialized view
- Changing the data type of a physical schema table column or MV column
- Changing the source of a materialized view or physical schema table, whether by selecting another source file in the Data Source properties dialog or editing the query
- Adding or changing a key column (changing the column function from key to dimension or measure and vice versa) in a physical schema table or MV
- Adding a new physical schema table column
- Adding a new MV column
- Changing the object type, for example, changing a physical schema table to an Incorta Analyzer table or MV
- Removing a physical schema table column or a materialized view column that functions as a key
- Changing the encryption status of one or more columns in a physical schema table or materialized view
Incorta does not recommend running concurrent schema model update jobs and load jobs on the same schema or dependent schemas as this may result in errors or inaccurate data.
Schema Designer actions for load jobs
With the Schema Designer, you can perform the following actions for data load jobs
- Change the load order for physical schema objects
- Perform a full load of the physical schema data
- Perform an incremental load of the physical schema data
- Load physical schema data from staging
- Create a load plan
- Perform a load of a physical schema object
- Review the status of a load job
Before the 2024.1.0 release, you could use the Schema Designer to create a scheduled job to send email notifications with the status of loading a schema. Starting with the 2024.1.x releases, load plan notifications have replaced schema load notifications, allowing for creating email notifications at the load plan level rather than the schema level. You can use the Scheduler to create load plan notifications.
Change the load order for physical schema objects
- In the Navigation bar, select Schema.
- In the list view of schemas, select a given physical schema.
- In the Schema Designer, select Settings (gear icon).
- Select Change Load Order.
- In the Choose Load Order dialog, you can do the following to manage the table groups before changing their load order:
- For each table (or materialized view) that you want to add to a new load group, select the check box, and then select Add to New Group.
- For each table (or materialized view) that you want to add to an existing load group, select the check box, and then select Add to Existing Group.
- To reset table grouping and order, select Revert to Default.
- To set the order of a table group, select the up arrow (to the right of the group name) to move the group upwards, or select the down arrow (to the right of the group name) to move the group downwards. Repeat this action until you order the table groups as required.
- Select Save.
Perform a full load of the physical schema data
- In the Schema Designer, in the Action bar, select Load → Full Load.
- In the dialog, select Load.
Perform an incremental load of the physical schema data
- In the Schema Designer, in the Action bar, select Load → Incremental Load.
- In the dialog, select Load.
Load physical schema data from staging
- In the Schema Designer, in the Action bar, select Load → Staging Load.
- In the dialog, select Load.
Create a load plan
You can organize schemas in a load plan into separate groups that the Loader Service handles sequentially in different load jobs. However, the Loader Service continues to handle schemas in a group concurrently in a single load job.
Here are the steps to add a load plan using the Schema Designer.
- In the Schema Designer, in the Action bar, select Load → Add Load Job.
- In the New Load Plan dialog, specify the load plan details, and then save the plan details.
- Set the schedule options to execute the load plan or select to Schedule later.
For information about the details of a load plan and the schedule options, refer to Tools → Scheduler → Create a scheduled load plan.
Scheduled load plans appear in the Scheduler, under Load Plans. A user with the appropriate access rights to the Scheduler can manage these plans, in addition to creating new plans.
Perform a load of a physical schema object
- In the Schema Designer, in the Tables tab, for a given physical schema object, select More Options (⋮ vertical ellipsis).
- Do one of the following:
- Select Load Table to start a full load job.
- Select Load from Staging to start a load-from-staging job.
- In the dialog, select Load.
Review the status of a load job
- In the Schema Designer, in the Summary section, select the Last Load Status link.
- In the Load Job Details Viewer, review the load job status.