Tools → Data Manager

About the Data Manager

The Data Manager allows a tenant user with the necessary access permissions to search for, create, and manage external data sources, local data folders, local data files, data agents, and data destinations.

Data Manager Access Permissions

A user that belongs to a group with the Schema Manager or the SuperRole role can access the Data Manager. To access the Data Manager for a given tenant, in the Navigation bar, select the Data tab.

Data Manager Anatomy

The Data Manager consists of the following:

• Action bar
• Context bar
• Search bar
• List view

Action bar

By default, the Action bar shows the context title, either External Data Sources or Data (for Local Data Files). For Local Data Files, the Action bar title is a navigable folder path.

+ New in the Action bar opens the Add New menu. The possible Add New menu options are:

• Add Data Source
• Add Data Destination
• Create Folder
• Add Data Agent

Context bar

The Context bar consists of three tabs: External Data Sources, Local Data Files, and Data Destinations. Tab selection determines the search context as well as the List view results.

Search bar

The Search bar contains a search text box. Search for any part of a given search term. When the Context bar tab selection is External Data Sources, the search returns a result list of zero or more external data sources. When the tab selection is Local Data Files, the search returns a result list of zero or more local data folders and files. When the tab selection is Data Agents, the search returns a result list of zero or more data agents. When the tab selection is Data Destinations, the search returns a result list of zero or more data destinations.

The Search bar shows:

• the number of items in the result list
• the number of selected items in the result list with the option (x) to deselect the items
• Share icon, if applicable, to open the Share dialog and assign or remove access right permission to the selected item. This icon is available only when you select one item.
• Delete icon, to delete all selected items

List view

The List view shows all results for a given search term and search context. By default, the list view only show items where the given user has view access permissions to the items.

The List view shows 20 items at a time. For lists with more than 20 items, the view contains page controls. The page controls allow the user to navigate up to 5 pages (100 items) forwards or backwards as well as to the first page or to the last page of items.

External Data Sources list view

External data source items appear in alphabetical, ascending order. For External Data Sources, the List view shows the following properties:

Local Data Files list view

By default, the Local data folder and file items appear descending order with last modified items on top. To sort the items by the column values in either ascending or descending order, select a specific column header

For a given folder item, you can select the row, and navigate into the folder. The Action bar title shows a navigable folder path.

For Local Data Files, the list view shows the following properties:

Data Agents list view

Data agent items appear in alphabetical, ascending order. For Data Agents, the List view shows the following properties:

Data Destinations list view

Data destination items appear in alphabetical, ascending order.

For Data Destinations, the List view shows the following properties:

Data Manager Actions for external data sources

With the Data Manager, you can perform the following actions for External Data Sources:

• Create an external data source
• View a list of external data sources
• Search for an external data source
• View data source data lineage
• Test an external data source
• Edit an external data source
• Share an external data source
• Delete an external data source
• Delete one or more external data sources

Create an external data source

Before creating an external data source, review References → Connectors → for specific property definitions, configurations, and implementation steps.

Here are the general steps to create an external data source:

• In the Navigation bar, select Data.
• In the Action bar, select + New → Add Data Source.
• In the Choose a Data Source dialog, select a data source connector from one the following categories:
  • Database
  • Application
  • File System
  • Query Service
  • Data Lake
  • Streaming Data
  • Other
  • Custom
• In the New Data Source dialog, specify the properties for the given data source connector.
• To test the connection, optionally select Test Connection.
• To save, select Ok.

View a list of external data sources

• In the Navigation bar, select Data.
• In the Context bar, select the External Data Sources tab.
• Review the list view of external data sources.

Search for an external data source

• In the Navigation bar, select Data.
• In the Context bar, select the External Data Sources tab.
• In the Search bar, enter a search term.
• Review the list view of external data sources.

View data source data lineage

• In the Navigation bar, select Data.
• In the Context bar, select the External Data Sources tab.
• In the list view of external data sources, for a given row, select Lineage.
• Use the Data Lineage Viewer v2 to review where the data source is utilized.

Test an external data source

• In the list view of external data sources, select a given row.
• Select Test Connection.

Edit an external data source

• In the list view of external data sources, select a given row.
• Select Edit.
• In the Edit Data Source dialog, make the necessary configuration updates.
• Optionally select Test Connection.
• To save your changes, select Ok.

Share an external data source

• In the list view of external data sources, select a given row.
• Select Share.

In the Share dialog, you can:

• remove an existing user or group.
• modify an existing access permission for user or group.
• add a new user or group with an access permission.

Delete an external data source

• In the list view of external data sources, select a given row.
• Select Delete (trash can).
• In the dialog, select Delete.

Delete one or more external data sources

• In the list view of external data sources, for each external data source that you want to delete, select the check box.
• In the Search bar, next to Selected, select Delete (trash can).
• In the dialog, select Delete.

Data Manager actions for local data folders and files

Using the Data Manager, you can perform the following actions for local data folders and files:

• Create and upload one or more local data folders and or local data files, including subfolders and files
• Create a local data folder
• View a list of local data folders and files
• Search for a local data folder or file
• Share a local data folder or file
• Delete a local data folder or file
• Delete one or more local data folders or files

Create and upload one or more local data folders and or local data files, including subfolders and files

A folder can contain zero or more files with zero or more subfolders. Incorta preserves the hierarchy of folders. Incorta only uploads files with the following supported file extensions. After upload, Incorta will unzip compressed folders and files.

Here are the steps to create and one or more local data folders and/or local data files, including subfolders and files:

• In the Navigation bar, select Data.
• In the Action bar, select + New → Add Data Source.
• In the Choose a Data Source dialog, in Data Files, select Upload Data Folder.
• In the Upload Data Folder dialog, in Upload Options, optionally select Overwrite existing file.
• Drag and drop one or more parent folders or one or more files to the Upload Data Folder dialog.

Create a local data folder

A local data folder can contain zero or more files with zero or more subfolders. To create a local data folder, follow these steps:

• In the Navigation bar, select Data.
• In the Action bar, select + New → Create Folder.
• In the Add Folder dialog, enter the local data folder Name.
• Select Add to create the folder.

View a list of local data folders and files

• In the Navigation bar, select Data.
• In the Context bar, select the Local Data Source tab.
• Review the list view of local data folders and files.

Search for a local data folder or file

• In the Navigation bar, select Data.
• In the Context bar, select the Local Data Source tab.
• In the Search bar, enter a search term.
• In the Search result dialog, select a local data folder or file.
• Review the list view of local data folders and files.

Share a local data folder or file

• In the list view of local data folder or file, select a given row.
• Select Share.

In the Share dialog, you can:

• remove an existing user or group.
• modify an existing access permission for user or group.
• add a new user or group with an access permission.

Delete a local data folder or file

• In the list view of local data folder or file, select a given row.
• Select Delete (trash can).
• In the dialog, select Delete.

Delete one or more local data folders or files

• In the list view of local data file, for each local data file or folder that you want to delete, select the check box.
• In the Search bar, next to Selected, select Delete (trash can).
• In the dialog, select Delete.

Data Manager Actions For Data Agents

You can utilize a data agent to securely ingest data from an external data source that is behind a firewall.

You create a data agent in the Data Manager to authenticate and monitor a remote data agent service. Only a Tenant Administrator (Super User) or user that belongs to a group with the SuperRole role for a given tenant can create a data agent instance that connects to a data agent service. A user that belongs to a group with the Schema Manager role can see a list of data agents in the Data Manager.

With the Data Manager, you can perform the following actions for Data Agents:

• Create a data agent
• View a list of data agents
• Search for a data agent
• Generate an authentication file
• Start a data agent
• Stop a data agent
• Restart a data agent
• Edit a data agent
• Delete a data agent
• Delete one or more data agents

Create a data agent

Here are the steps to create a data agent in the Data Manager:

• Sign in to the tenant as the Tenant Administrator (Super User) or user that belongs to a group with the SuperRole role.
• In the Navigation bar, select Data.
• In the Action bar, select + New > Add Data Agent.
• In the Create Data Agent dialog, enter a Data Agent Name and optionally enter a description.
• In the Generate Authentication File, select Generate Later or Generate Now.

View a list of data agents

• In the Navigation bar, select Data.
• In the Context bar, select the Data Agents tab.
• Review the list view of data agents.

Search for a data agent

• In the Navigation bar, select Data.
• In the Context bar, select the Data Agents tab.
• In the Search bar, enter a search term.
• Review the list view of data agents.

Generate an authentication file

You can generate or regenerate an authentication file from the Data Manager for a given data agent. The file contains all the information needed by the data agent service to connect to the Incorta cluster. The file includes information regarding the various hosts, ports, and TLS/SSL certificates.

• In the Navigation bar, select Data.
• In the Context bar, select the Data Agents tab.
• In the list view of data agents, for a given data agent row, select Generate Authentication File or Regenerate Authentication File.
• As required, move and copy the downloaded .auth file.

Start a data agent

• In the list view of data agents, for a given row, select Start.

Stop a data agent

• In the list view of data agents, for a given row, select Stop.

Restart a data agent

• In the list view of data agents, for a given row, select Restart.

Edit a data agent

• In the list view of data agents, select a given row.
• Select Edit.
• In the Edit Data Agent dialog, you can edit the Description.
• To save your changes, select Ok.

Delete a data agent

• In the list view of data agents, select a given row.
• Select Delete (trash can).
• In the dialog, select Delete.

Delete one or more data agents

• In the list view of data agents, for each data agent that you want to delete, select the check box.
• In the Search bar, next to Selected, select Delete (trash can).
• In the dialog, select Delete.

Data Manager Actions For Data Destinations

You can utilize dashboard destinations to export supported insights to Google Drive, Google Sheets, Microsoft OneDrive, FTP servers, or SFTP servers. You can also use schema destinations, such as Synapse and BigQuery, to send schemas

For more information, refer to Concepts → Data Destination

With the Data Manager, you can perform the following actions for Data Destinations:

• Create a data destination
• View a list of data destinations
• Search for a data destination
• Test a data destination
• Edit a data destination
• Share a data destination
• Delete a data destination
• Delete one or more data destinations

Create a data destination

Here are the general steps to create a data destination:

• In the Navigation bar, select Data.
• In the Action bar, select + New → Add Data Destination.
• In the Choose a Data Destination dialog, select a data destination type from the following:
  • FTP
  • Google Drive
  • Google Sheets
  • OneDrive
  • SFTP
  • Blackline
  • BigQuery
  • Synapse Analytics
• In the New Data Destination dialog, specify the properties for the given data destination type.
• To test the connection, select Test Connection.
• To save, select Ok.

For information about the available data destination properties per type, refer to Concepts → Data Destination → Data destination properties

View a list of data destinations

• In the Navigation bar, select Data.
• In the Context bar, select the Data Destinations tab.
• Review the list view of data destinations.

Search for a data destination

• In the Navigation bar, select Data.
• In the Context bar, select the Data Destinations tab.
• In the Search bar, enter a search term.
• Review the list view of data destinations.

Test a data destination

• In the list view of data destinations, select a given row.
• Select Test Connection.

Edit a data destination

• In the list view of data destinations, select a given row.
• Select Edit.
• In the Edit Data Destination dialog, make the necessary configuration updates.
• Optionally, select Test Connection.
• To save your changes, select Ok.

Share a data destination

• In the list view of data destinations, select a given row.
• Select Share.

In the Share dialog, you can:

• remove an existing user or group.
• modify an existing access right for a user or group.
• add a new user or group with an access right.

Delete a data destination

• In the list view of data destinations, select a given row.
• Select Delete (trash can).
• In the dialog, select Delete.

Delete one or more data destinations

• In the list view of data destinations, for each data destination that you want to delete, select the check box.
• In the Search bar, next to Selected, select Delete (trash can).
• In the dialog, select Delete.