Concepts → Shared Storage
About Shared Storage
For a standalone instance of Incorta, shared storage is local to the host that runs the Incorta Node. In other words, shared storage is the local file system.
For an Incorta cluster, where there are multiple Incorta Nodes with each node running at least one Analytics or Loader Service, it is not possible to share a local file system among the various hosts. Instead, a System Administrator must implement a common mount for shared storage for each host. An on-premises Incorta cluster typically mounts a Network File System v4 (NFSv4) as a directory common to all the hosts in a cluster. From the perspective of the host, the mounted directory appears a local file system directory.
A cloud deployed Incorta cluster typically also has a mounted directory for shared storage which is similar to NFSv4. For example, a cluster deployed in AWS on EC2 instances, typically mounts shared storage as an Elastic File Share (EFS) directory.
For a given tenant, shared storage best characterizes the various usages of a local file system, network file system, or similar mount such as EFS.
Here are some of various ways that Incorta as a direct data platform uses shared storage:
- Users that belong to a group with the Schema Manager role can upload one or more files and folders
- The Loader Service creates direct data mapping and parquet files
- The Analytics Service reads direct data mapping and parquet files
- Apache Spark both writes and reads parquet files
- When enabled, an Apache Kafka external data source ingests data from a Kafka broker for given topic and writes encrypted files