1. Home
  2. On-premises 5.2
  3. Guides
    1. Start
    2. Administer
    3. Analyze
    4. Configure
      1. Configure Server
      2. Configure Tenants
      3. Email Configuration
      4. Apache Web Server Configuration
      5. Zookeeper Cluster Configuration
      6. Performance Tuning
      7. Optimize Performance
      8. Memory Usage
      9. Configure High Availability
      10. Memory Configuration
    5. Deploy
    6. Install
    7. Migrate
    8. Secure
    9. Upgrade
    10. Content Manager
  4. Tours
  5. Concepts
  6. Tools
  7. References
  8. Integrations
  9. Data applications
  10. Releases

Memory Configuration

You can configure the Incorta memory space after installation. By default, Incorta configures a memory space in the start.sh file. In that file Incorta configures the maximum memory that Java allocates and then manages the memory inside of that instance.

Understanding Memory

To review the memory used by Incorta and the various Incorta users use the top command to list the memory contents.

top -a

The result is a list of all users and their memory resources. Of interest here is:

  • Process IDs
  • User associated with the PID
  • VIRT which is the virtual memory consumed
  • RES which is the residual or remaining memory

More information is available about the process by grepping for the process ID.

Ps -ef | grep 24813

The result displays the memory configuration along with the location of the home directory.

In this case, -Xms is the minimum amount of memory and -Xmx is the maximum. Note that the startup script only calls for Java to allocate the memory. Incorta handles memory management. Note also that you can have multiple Incorta instances with different process IDs.

You can further drill into memory and its configuration using the Tomcat server status tool. This tool shows the memory pool which matches the Xms and Xmx numbers, and the memory allocation by process.

Select Server Status to drill into the processes and their memory settings.

The PS Eden space is the working space for loading Incorta data. PS Old Glen is the Incorta data memory pool. When data usage reaches 80 to 90 percent you are running out of memory.

Note: The Tomcat manager is an option. It is not installed by default. Incorta recommends only installing the Tomcat manager inside of your firewall, for dev and UAT environments only. Incorta does not recommend installing this outside of your firewall or on your production environment.

The initial memory allocation is determined during installation. The amount of memory needed by your implementation is determined by the amount of data you have. During the planning stage, configure for 35GB plus the amount of data you have.

Changing the Memory Settings

The memory settings can be modified in the startup.sh file by modifying the -Xms and -Xmx settings line in the start.sh (start.bat for Windows) file located at ~/IncortaAnalytics/IncortaNode/:

export JAVA_OPTS="-Xms20480m Xmx20480m -Dfile.encoding=UTF-8"

Memory Sizing

The Incorta Loader Service compresses the data on ingestion. Typical compression rates vary between 1-3x, depending on the cardinality of the data. In addition to the data memory requirements, the Incorta Direct Data Platform requires 30 percent working memory and indices. With Oracle E-Business Suite data, we recommend a conservative compression rate of 1.5x, which includes all the required memory for running Incorta Analytics.

The following table provides a sample of an Oracle E-Business Suite environment and sizing information for the Incorta Direct Data Platform E-Business Suite Accelerators. Note that only columns with analytical value are imported, not the full tables.

Table NameSource Size (GB)Incorta Size (GB)Rows (M)
RA_CUSTOMER_TRX_ALL2.51.413
RA_CUSTOMER_TRX_LINES_ALL7123.9471
RACUST_TRX_LINE_GL_DIST ALL2520300
HZ_PARTIES147.390

The Incorta Loader Service stores data on disk in a compressed format with compression rates between 1-10x, and 2x for EBS data. A High Availability Incorta Cluster configuration requires double the disk space.

Strategies to Reduce Memory Usage

The loader service loads only formula columns and columns used in joins into the loader service memory. The loader service does not load other columns into the loader service memory or create snapshot files. The analytics service reads the data columns directly from parquet. Tables with load filters and encryption are still loaded into memory for the analytics service to create a snapshot file.

The following diagram illustrates the differences between the way Incorta used memory in previous releases and the way it uses memory in this release.

You can also:

  • Enable dynamic memory allocation on the loader service to remove unused columns in memory. Evict columns from loader service memory, after writing to snapshots, if memory is tight on the loader service.
  • Set up analytics service column warmup. After you restart the analytics service, dashboard queries can load slowly. To increase the time to load dashboard queries after you restart the analytics service, you can choose to load and warm up specific columns first. You can choose one of the following column warmup strategies:
    • Business view columns: Load all columns referenced in business schema views
    • None: Don’t pre-load the columns. Only load on demand. None works best for small deployments with ad-hoc queries
    • Last used columns: Load the previous state prior to shut down.
    • All: Pre-load all columns into memory. All works best when you need to support ad-hoc queries, if there are no business schemas in place, and when the time between analytics service startup and dashboard usage is significant.