Connectors → Oracle Cloud Applications

About Oracle Cloud Applications

Oracle Cloud Applications offer both an integrated suite of cloud applications and the cloud infrastructure platform. Your Oracle Cloud Applications allow you to build and extend your applications and better cope with any process changes.

The Oracle Cloud Applications connector supports the following Incorta specific functionality:

Feature	Supported
Chunking
Data Agent
Encryption at Ingest
Incremental Load	✔
Multi-Source	✔
OAuth
Performance Optimized	✔
Remote
Single-Source	✔
Spark Extraction
Webhook Callbacks	✔

Steps to connect Oracle Cloud Applications and Incorta

To connect Oracle Cloud Applications and Incorta, here are the high level steps, tools, and procedures:

Create an external data source
Create a schema with the Schema Wizard
or,Create a schema with the Schema Designer
Load the schema
Explore the schema

Create an external data source

Here are the steps to create a external data source with the Oracle Cloud Applications connector:

Sign in to the Incorta Direct Data Platform™.
In the Navigation bar, select Data.
In the Action bar, select + New → Add Data Source.
In the Choose a Data Source dialog, in Application, select Oracle Cloud Applications.
In the New Data Source dialog, specify the applicable connector properties.
To test, select Test Connection.
Select Ok to save your changes.

Oracle Cloud Applications connector properties

Here are the properties for the Oracle Cloud Applications connector:

Property	Control	Description
Name Your Data Source	text box	Enter a name for the data source
Username	text box	Enter the Oracle Cloud Applications account identifier.
Password	text box	Enter the Oracle Cloud Applications password.
Oracle Cloud Applications URL	text box	Data location For example, if you set the cluster path prefix to /mycluster when you created a service instance, then your URL should look like: `https://public_ip:port/mycluster/app`
Root Query Text	text box	An optional filtering query that can be used to filter files retrieved from WCC. Example: dSecurityGroup `<matches>` OBIAIMport
Data Type Discovery Policy	text box	Defines the metadata definition files. These files must be uploaded first to Incorta Data Files, and must have *.csv extension. Options: Use Both Sample data and Metadata Definition Files Use Sample data
Metadata Directory	text box	Leave blank if metadata files uploaded in the root data directory, otherwise enter path of metadata directory, for example: /path/to/metadata/directory
File Name Pattern	text box	By default the file name pattern looks like this example: `file_<catalog>_<schema>_<table>batch.zip` You can enter an irregular file name pattern, but you must toggle the irregular pattern property in the data source properties before loading
Load Only Files Created Since	text box	Acts as a time filter for all the results concerning this data source. You can either use a fixed date, or a look-back period(number of years, months, or days). Fixed date format: yyyy-MM-dd HH:mm. Example: 2019-05-31 15:30. Look-back period example: 90d.
Keep Each File For	drop-down-list	Enable Keep Cached Files to configure this property. Choose the required cache time period for each extracted file, by default the cache is cleared every hour.
File Extension	drop-down-list	Choose the appropriate file extension wither CSV or pecsv
Default Hour Of The Day For Date Columns	text box	Enter the default hour of the day for date columns.
Advanced Options	toggle	Enable this property to use the advanced connection options
Request Page Size	text box	Enable Advanced Options to configure this property. Enter the maximum number of results to return per page from Oracle Cloud Applications to enhance discovery performance. Setting a higher value may result in better performance at the cost of additional memory consumed per page. The default is 2000.
Maximum Cache Size	text box	Enter the maximum cache size to use on disk for downloaded files. The default is 250 GB.
Extra Options	text box	Available starting with 5.2.10. Enter supported extra options in the form of `key=value`. For example, to set the data source time zone to UTC, enter `Timezone=UTC`.

Supported extra options

Option	Description	Example	Release
Timezone	Specify the time zone of your Oracle Cloud Applications data source if it is different from the time zone of your Incorta cluster to make sure that time data values are accurately written in Parquet in Coordinated Universal Time (UTC). Set the time zone value using the Internet Assigned Numbers Authority (IANA) standard time zone names or abbreviations.	`Timezone=UTC`	5.2.10

Create a schema with the Schema Wizard

Here are the steps to create an Oracle Cloud Applications schema with the Schema Wizard:

Sign in to the Incorta Direct Data Platform™.
In the Navigation bar, select Schema.
In the Action bar, select + New → Schema Wizard.
In (1) Choose a Source, specify the following:
- For Enter a name, enter the schema name.
- For Select a Datasource, select the Oracle Cloud Applications data source.
- Optionally create a description.
In the Schema Wizard footer, select Next.
In (2) Manage Tables, in the Data Panel, first select the name of the Data Source, and then check the Select All checkbox.
In the Schema Wizard footer, select Next.
In (3) Finalize, in the Schema Wizard footer, select Create Schema.

Create a schema with the Schema Designer

Here are the steps to create an Oracle Cloud Applications schema using the Schema Designer:

Sign in to the Incorta Direct Data Platform.
In the Navigation bar, select Schema.
In the Action bar, select + New → Create Schema.
In Name, specify the schema name and select Save.
In Start adding tables to your schema, select Oracle Cloud Applications.
In the Data Source dialog, specify the Oracle Cloud Applications table data source properties.
Select Add.
In the Table Editor, in the Table Summary section, enter the table name.
To save your changes, select Done in the Action bar.

Oracle Cloud Applications table data source properties

For a schema table in Incorta, you can define the following Oracle Cloud Applications specific data source properties as follows:

Property	Control	Description
Type	drop down list	The default is Oracle Cloud Applications
Data Source	drop down list	Select the Oracle Cloud Applications external data source
Data Store Entry Method	drop down list	You can either select to Enter Fully Qualified Data Store Name or Select from List.
Top-Level Application Module	drop down list	Choose Select from List in Data Store Entry Method to configure this property. Select your catalogue name.
Intermediate Application Module	drop down list	Choose Select from List in Data Store Entry Method to configure this property. Select your schema name.
Data Store	text box	Enter Fully Qualified Data Store name.
Incremental	toggle	Enable the incremental load configuration for the schema table
Irregular Pattern	toggle	Toggle if Fully Qualified Data Store name does not follow conventional pattern of `file_<schema>_<catalog>_<table>` or if it contains uppercase letters
Which Batches to Load	drop down list	For full-load-always tables, control which batches you want to load into Incorta. Select one of the options: ● Load Last Batch Only (default) ● Load All Batches ● Load Last Valid Batch Only (available starting 5.2.9)*
Data Type Discovery Policy	drop down list	Metadata Definition files must be uploaded first to Incorta data files, and must have `.csv` extension.
Load Only Files Created Since	text box	Filter which files to load based on the file creation timestamp. Use either a fixed date in the format `yyyy-MM-dd HH:mm` (example: `2019-05-31 15:30`), or a look-back period expressed as the number of years, months or days before the loading starts (example: use the value 90d to load files that have been created since 90 days)
Date Format	drop down list	Choose the suitable date format from the list provided. Or, you can choose Other and provide your own format in the Other field that will appear.
Timestamp Format	drop down list	Choose the suitable timestamp format from the list provided. Or, you can choose Other and provide your own format in the Other field that will appear.
Character Set	drop down list	Select the character set
Callback	toggle	Enable this option to call back on the source data set
Callback URL	text box	Enable Callback to configure this property. Specify the URL.

Note

The Load Last Valid Batch Only is available starting with 5.2.9 to allow the connector to search for the last valid batch in the batches that are uploaded to UCM within a defined period from the load time. The default period is 7 days; however, you can set a different period by editing the Loader Service's service.properties file and setting a value for the oracle.cloud.application.batch.lookup.period property. If no valid batch is found, the load job will fail with an error message: Couldn't find a valid record set.

View the schema diagram with the Schema Diagram Viewer

Here are the steps to view the schema diagram using the Schema Diagram Viewer:

Sign in to the Incorta Direct Data Platform™.
In the Navigation bar, select Schema.
In the list of schemas, select the Oracle Cloud Applications schema.
In the Schema Designer, in the Action bar, select Diagram.

Load the schema

Here are the steps to perform a Full Load of the Oracle Cloud Applications schema using the Schema Designer:

Sign in to the Incorta Direct Data Platform™.
In the Navigation bar, select Schema.
In the list of schemas, select the Oracle Cloud Applications schema.
In the Schema Designer, in the Action bar, select Load → Load Now → Full.
To review the load status, in Last Load Status, select the date.

Explore the schema

With the full load of the Oracle Cloud Applications schema complete, you can use the Analyzer to explore the schema, create your first insight, and save the insight to a new dashboard.

To open the Analyzer from the schema, follow these steps:

In the Navigation bar, select Schema.
In the Schema Manager, in the List view, select the Oracle Cloud Applications schema.
In the Schema Designer, in the Action bar, select Explore Data.

Additional Considerations

Types of Incremental Loads

You can enable Incremental Load for a Oracle Cloud Applications data source. There are two types of incremental extracts:

Last Successful Extract Time

Fetch updates since the last time the tables were loaded. It is determined by the difference between the current time and the database timestamp.

Maximum Value of a Column

The column-based strategy depends on an extra column called Incremental Column in each table. The Oracle Cloud Applications connector supports both timestamp and numeric columns. A timestamp column is of the type date or timestamp. A numeric column is of the type int or long.

Note

Changing the incremental load strategy requires a full load to ensure data integrity.