Tools → Schema Manager → Schemas

Schemas list view

By default, the Schema items appear in alphabetical, ascending order. The list shows 40 items at a time. To sort the items by the column values in either ascending or descending order, select a specific column header. For any schema row, select any column other than the Status and Last Load links, to open the item in the Schema Designer. The Schemas tab shows the size of the data on disk. The Refresh (circle with arrow icon) is available to refresh the list view and recalculate the data size.

When you select one or more physical schema, the Search bar shows the More Options button (⋮ vertical ellipsis). The available options are:

• Export: use to export the selected physical schemas to an Incorta package
• Schedule Load Plan: use to schedule a load plan for the selected physical schema
• Create Email Notification: use to create an email notification for the selected physical schemas (not available starting 2024.1.0)
• Delete: use to delete all selected physical schemas

For Schemas, the List view shows the following properties:

Schema Manager actions for physical schemas

With the Schema Manager, you can perform the following actions for physical schemas:

• Create a physical schema with the Schema Wizard
• Create a physical schema with the Schema Designer
• Import a physical schema
• View a list of physical schemas
• Search for a physical schema
• Filter physical schemas by the load status
• Filter physical schemas by the model update status
• Edit a physical schema
• View physical schema details
• View the load job details for a physical schema
• View the physical schema update jobs
• View blocking jobs
• Add or edit a physical schema description
• Optimize performance for physical schema objects
• Change the load order of physical schema objects
• Schedule the load of a physical schema
• Schedule the load of one or more physical schemas
• Share a physical schema
• Export a physical schema
• Export one or more physical schemas
• Create physical schema load email notifications
• Delete a physical schema
• Delete one or more physical schemas

Schema Manager access rights matrix

As the Super User or a user that belongs to a group with the Schema Manager or the SuperRole role, you can access the Schema Manager. You can import physical schemas and create them, whether with the Schema Wizard or the Schema Designer. However, to be able to view, search for, or manage items on this page, you need to have the suitable access rights to them.

The following table shows the required access rights for each action.

Create a physical schema with the Schema Wizard

Here are the steps to create a physical schema with the Schema Wizard:

• On the Navigation bar, select Schema.
• On the Action bar, select + New → Schema Wizard.
• In (1) Choose a Source, specify the following:
  • In Enter a name, enter the physical schema name.
  • In Select a Datasource, select the external data source.
  • Optionally, create a description.
• In the Schema Wizard footer, select Next.
• In (2) Manage Tables, in the Data panel, select the name of the Data Source, and then select the tables, files, or worksheets you want or select the Select All checkbox to select all tables, files, or worksheets depending upon the data source you selected.
• In the Schema Wizard footer, select Next.
• In (3) Finalize, in the Schema Wizard footer, select Create Schema.

Create a physical schema with the Schema Designer

Before creating a physical schema with the Schema Designer, review References → Connectors → for specific table property definitions.

Here are the steps to create a physical schema using the Schema Designer:

• On the Navigation bar, select Schema.
• On the Action bar, select + New → Create Schema.
• In Name, specify the physical schema name, and select Save.
• In Start adding tables to your physical schema, select one of the following:
  • Schema Wizard: to use the Schema Wizard to select the tables, files, or worksheets you want to add
  • Incorta Analyzer: to use the Analyzer to create a table or aggregated table insight as a physical schema Incorta Analyzer table
  • Incorta SQL: to use Incorta SQL Engine to create an Incorta SQL table by querying objects in the same or other physical schemas. The
  • SQL Table: to create a SQL table from different data source types, such as MySQL, Oracle, IBM DB2, Cassandra (Simba), NetSuite Suite Analytics, Presto, SAP Hana, and BigQuery
  • File System: to create a table from a local data file
  • Cassandra: to create a table from an Apache Cassandra data source
  • Cisco Meraki: to create a table from a Cisco Meraki data source
  • Data Lake: to create a table from a Data Lake data source
  • DynamoDB: to create a table from an AWS DynamoDB data source
  • Essbase: to create a table from an Oracle Essbase data source
  • Freshservice: to create a table from a Freshservice data source
  • Google Sheets: to create a table from a Google Sheets data source
  • Kafka: to create a table from an Apache Kafka data source
  • NetSuite: to create a table from a NetSuite Web Service data source
  • OneLogin: to create a table from a OneLogin data source
  • Oracle Cloud Applications: to create a table from an Oracle Cloud Application data source
  • SalesForce: to create a table from a Salesforce data source
  • ServiceNow: to create a table from a ServiceNow data source
  • SharePoint: to create a table from a SharePoint data source
  • Splunk: to create a table from a Splunk data source
  • Zuora: to create a table from a Zuora data source
  • Materialized: to create a Materialized View
  • Alias: to create an alias that references a selected physical schema object, in the same physical schema or other physical schemas
• In the Data Source dialog, specify the properties for the given object.
• Select Add.
• In the Table Editor, in the Table Summary section, enter the physical schema object name.
• To save your changes, on the Action bar, select Done.

Import a physical schema

You can import one or more physical schemas that exist in the same .zip file. When you import a physical schema, if there is no physical schema that shares the same name with the imported one, a new physical schema is created from this imported one with no version. If you try to import a physical schema that shares the name with an existing one, the import fails. However, you can overwrite this existing physical schema. When you overwrite a physical schema that has saved versions (whether named or not), these versions are not deleted, and a new saved version is added as the current version.

Here are the steps to import a physical schema:

• On the Navigation bar, select Schema.
• On the Action bar, select + New → Import Schema.
• In the Import Schema dialog, optionally select the Overwrite existing schema checkbox to overwrite any existing physical schema that shares the same name with the imported one, if any.
• Do one of the following:
  • Drag the .zip file that contains the physical schema(s) that you want to import to the Import Schema dialog.
  • Select Click or drag a file here to upload to select the .zip file, and then select Open.
• Wait until the physical schema(s) and scheduled load job(s), if any, are imported.
• In the Import Results dialog, review the status of the import operation.
• Select Close.

The following are the details in the Import Results dialog:

View a list of physical schemas

• On the Navigation bar, select Schema.
• On the Schemas tab, review the list view of physical schemas.

Search for a physical schema

• On the Navigation bar, select Schema.
• On the Schemas tab, on the Search bar, enter a search term.
• Review the list view of physical schemas.

To reset the search, on the Search bar, in the search box, select the remove button (x).

Filter physical schemas by the load status

• On the Navigation bar, select Schema.
• On the Schemas tab, in the list view header row, select the filter icon next to Load Status.
• Select one or more load statuses to filter the physical schemas list view. The load status options are:
  • Succeeded
  • Finished With Errors
  • Failed
  • Interrupted
  • In Progress (Running or in Planning)
  • In Queue
• Select OK.

Reset the load status filter

• On the Navigation bar, select Schema.
• On the Schemas tab, in the list view header row, select the filter icon next to Load Status.
• Select Reset.

Filter physical schemas by the model update status

• On the Navigation bar, select Schema.
• On the Schemas tab, in the list view header row, select the filter icon next to Model Status.
• Select one or more model update statuses to filter the physical schemas list view. The model update status options are:
  • Succeeded
  • Finished With Errors
  • Failed
  • Interrupted
  • In Progress
  • In Queue
• Select OK.

Reset the model update status filter

• On the Navigation bar, select Schema.
• On the Schemas tab, in the list view header row, select the filter icon next to Model Status.
• Select Reset.

Edit a physical schema

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for a given physical schema that you own or have edit access rights to, select any part of the row except the Status and Last Load links.
• In the Schema Designer, you can do the following:
  • view a list of the physical schema objects and joins
  • add, edit, or remove physical schema objects
  • add, edit, or remove joins between the physical schema objects
  • load data for the physical schema
  • load data for a selected physical schema object
  • view or manage load jobs
  • open the Table Editor for a selected physical schema object
  • switch to the Diagram view
  • open the Join Editor for a select join
  • explore physical schema data
  • add or edit a physical schema description
  • optimize performance for the physical schema objects
  • change the load order of the physical schema objects
  • schedule the load of a physical schema
  • share a physical schema
  • create physical schema load email notifications

View physical schema details

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for a given physical schema, select any part of the row except the Status and Last Load links.
• In the Schema Designer, you can do the following:
  • view load jobs
  • view physical schema objects and joins
  • view physical schema object details, such as the columns, formula columns, and load filters
  • explore physical schema data
  • view the physical schema diagram
  • view or edit the share options for the physical schema depending upon the access rights
  • view the physical schema description, objects load order, and performance optimization settings for the physical schema objects
  • create a physical schema load email notification

For more information, refer to Tools → Schema Designer

View the load job details for a physical schema

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for a given physical schema, select the Load Status or Last Load link.
• In the Load Job Viewer, review the details of the load jobs for the physical schema.

View the physical schema update jobs

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for a given physical schema, select the Model Status link.
• In the Model Update Viewer, review the details of the physical schema update jobs.

View blocking jobs

When an update job or a load job is in the commit phase or the load and post-load phase, respectively, it blocks the update and load jobs on dependent physical schemas. A dependent physical schema is one that has a cross join relationship with another physical schema. The blocked update jobs stay in the In Queue state till all blocking jobs are completed while the load jobs stay in the Load state. Starting with the 2021.4.1 release, as a schema manager user, you can view a list of jobs that block update jobs in the Schema Manager and the Model Update Viewer. This list shows the blocking physical schema, the type of the blocking job, load or update, and the current status of each blocking job. You can also access the details of the blocking job in the Load Job Viewer or the Model Update Viewer according to the type of the blocking job.

Here are the steps to view the list of blocking jobs of an update job in the Schema Manager:

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for a given physical schema that has a blocked update job, in Model Status, select the In Queue link. A pop-up dialog appears with a list of the blocking jobs.
• To view the details of a blocking job, in the pop-up dialog, select See Job Details.

Add or edit a physical schema description

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for a given physical schema row, select More Options (⋮ vertical ellipsis).
• Select Edit Description.
• In the Edit Description dialog, enter or update the physical schema description. You can enter up to 500 alphanumeric characters, including spaces and special characters.
• Select Save.

Optimize performance for physical schema objects

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for a given physical schema row, select More Options (⋮ vertical ellipsis).
• Select Performance Optimization.
• In the Optimize Table Performance dialog, in the objects list, for each object you want to optimize the performance, select the checkbox, and for each object that you do not want to optimize the performance, clear the checkbox.
• Select Save.

Change the load order of physical schema objects

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for a given physical schema row, select More Options (⋮ vertical ellipsis).
• Select Change Load Order.
• In the Choose Load Order dialog, you can do the following to manage the table groups before changing their load order:
  • For each table or Materialized View that you want to add to a new load group, select the checkbox, and then select Add to New Group.
  • For each table or Materialized View that you want to add to an existing load group, select the checkbox, and then select Add to Existing Group.
  • To reset table grouping and order, select Revert to Default.
• To set the order of a table group, select the up arrow (to the right of the group name) to move the group upwards, or select the down arrow (to the right of the group name) to move the group downwards. Repeat this action until you order the table groups as required.
• Select Save.

Schedule the load of a physical schema

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for a given physical schema row, select More Options (⋮ vertical ellipsis).
• Select Schedule.
• In the Add Schema Load dialog, in Schema Load Name, enter the name of the scheduled load job. You can enter up to 255 alphanumeric characters, including spaces, special characters, and emojis.
• In Description, optionally enter a description for the scheduled load job. You can enter up to 255 alphanumeric characters, including spaces, special characters, and emojis.
• The Schema box shows the selected physical schema.
• In Load Type, select the load type. The available options are:
  • Full
  • Incremental (default)
  • Staging
• In the Every section, specify the frequency and recurrences of this job.
  • Enter a number.
  • Select the time measure unit: minutes, hours, days, weeks, or months.
  • Select the No recurrence checkbox to run this job only once.
• In the Starting At/At section, specify the time to run this job. Select the time and time zone.
• Specify the validity period of the scheduled load job:
  • If the job is recurring, in the From section, do one of the following:
    • Select the start date and keep the Does not end checkbox selected to run it forever.
    • Select the start date, clear the Does not end checkbox, and select the end date to run the job during this period.
  • If the job is not recurring, in the On section, select the date to run this job.
• Select Add.

Schedule the load of one or more physical schemas

In releases before 2022.11.0, you can schedule the load of one schema only. The 2022.11.0 release introduced the multi-schema load plan preview feature. You can now schedule a plan for loading multiple schemas. The Loader Service treats a load plan as a single load job regardless of the number of physical schemas in the plan.

Here are the steps to schedule a load plan of one or more physical schemas using the Schema Manager:

• On the Navigation bar, select Schema.
• In the list view of physical schemas, do one of the following:
  • For a given physical schema row, select More Options (⋮ vertical ellipsis), and then select Schedule Load Plan.
  • Select the checkbox of one or more physical schemas. On the Search bar, select More Options (⋮ vertical ellipsis), and then select Schedule Load Plan.
• In the Add Load Plan dialog, specify the load plan details.
• Select Add.

For information about the details of a load plan, refer to Tools → Scheduler → Create a scheduled load plan.

Share a physical schema

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for a given physical schema row, select More Options (⋮ vertical ellipsis).
• Select Share.
• In the Share dialog, you can do the following:
  • remove an existing user or group
  • modify the existing access rights for a user or group
  • add a new user or group with the required access rights

Export a physical schema

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for a given physical schema row, select More Options (⋮ vertical ellipsis).
• Select Export.
• In the Export Schema(s) dialog, optionally select the Include scheduled jobs/Include load plans checkbox to include scheduled load jobs or load plans related to the selected physical schema in the export operation.
• Select Export. The exported physical schema .zip file is downloaded to the Downloads folder.

Export one or more physical schemas

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for each physical schema that you want to export, select the checkbox.
• On the Search bar, next to Selected, select More Options (⋮ vertical ellipsis).
• Select Export.
• In the Export Schema(s) dialog, optionally select the Include scheduled jobs/Include load plans checkbox to include scheduled load jobs or load plans related to the selected physical schema(s) in the export operation.
• Select Export. The exported physical schema(s) .zip file is downloaded to the Downloads folder.

Create physical schema load email notifications

Create an email notification for a physical schema

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for a given physical schema row, select More Options (⋮ vertical ellipsis).
• Select Create Email Notification.
• In the Create Email Notification dialog, enter the details of the email notification. For more information, see Email notification details.

Create an email notification for one or more physical schemas

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for each physical schema that you want to include in the email notification, select the checkbox.
• On the Search bar, next to Selected, select More Options (⋮ vertical ellipsis).
• Select Create Email Notification.
• In the Create Email Notification dialog, enter the details of the email notification. For more information, see Email notification details.

Email notification details

When you create an email notification for a given physical schema or multiple physical schemas, in the Create Email Notification dialog, enter the following:

• In Notification Name, enter the name of the new email notification. The length of the notification name must be between 1 and 255 alphanumeric characters, including spaces and special characters.
• In the Notify Upon section, specify the load job statuses that trigger sending the notification email. Available options are: Success, Failure, Finishing With Errors, and Interruption.
  Note

  Before 2023.1.0, you have only two options: Success and Failure. Jobs that finished with errors or were interrupted were considered failed jobs when sending the notification. There was no differentiation between the three cases then.

• The Select Schema(s) section, by default, shows the physical schemas that you selected. You can do the following:
  • In Search any schema name, enter a search term to search for specific physical schemas, and then in the search result list, select the physical schema(s) you want.
  • Select the Select All checkbox to include in the email notification all physical schemas you have access to or all physical schema(s) in the search result list.
  • In the physical schemas list or search result list, for each physical schema that you want to include in the email notification, select the checkbox.
• In the Recipients section, to add a recipient, in Search Names, Emails, and Groups, do one of the following:
  • To add a user or group of users, enter a search term to search for users or groups in the tenant, and then select the user or group from the search result list.
  • To add an external recipient, enter the email address, and then select the external recipient.
• In the Recipients list, for each recipient, select V (down arrow) to the right of the recipient name or email, and then select where the given recipient should go to: the To list, Cc list, or Bcc list.
• Optionally, in Body, enter the email body text. The body text you enter appears in the notification email before an auto-generated text. You can enter up to 4000 alphanumeric characters, including spaces and special characters.
• Select Done.

Delete a physical schema

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for a given physical schema row, select More Options (⋮ vertical ellipsis).
• Select Delete.
• In the dialog, select Delete.

Delete one or more physical schemas

• On the Navigation bar, select Schema.
• In the list view of physical schemas, for each physical schema that you want to delete, select the checkbox.
• On the Search bar, next to Selected, select More Options (⋮ vertical ellipsis).
• Select Delete.
• In the dialog, select Delete.