OCIFS Utility

The OCIFS utility gives you the ability to mount Oracle Cloud Infrastructure Object Storage buckets as file systems. You can then manage the objects in the buckets as regular files and directories.

Installing the OCIFS Utility

Install the OCIFS utility on systems or instances with Oracle Linux 8, or later.

Prerequisites:

  • OCI Object Storage: Ensure the system, or instance, on which you plan to install OCIFS has access to, or can connect to, an existing Object Storage bucket that you want mounted. The bucket can be empty or already populated with objects. For more information about Object Storage, see Overview of Object Storage. For information on how to create an Object Storage bucket, see Creating a Bucket.

  • Local Disk Space: OCIFS consumes local disk space for caching Object Storage objects. OCIFS caches objects of a mounted Object Storage bucket in a directory that you can specify. For best performance, designate a local directory for caching that's on a storage device with fast access time.

    Also, ensure that the cache directory is on a storage device with enough free space to host the entire contents of the mounted Object Storage bucket path, or, at minimum, all objects in the bucket to be read or written.
    Note

    OCIFS operations might fail if the storage device where the cache directory is located gets full. For more information on OCIFS caching, see OCIFS Cache Options.
  • CLI Configuration file: By default, OCIFS uses the API key-based authentication method to authenticate Object Storage. This authentication method requires a CLI configuration file that contains OCIFS-specific entries. For information on how to set up the configuration file for OCIFS, see API Key-Based Authentication.

To install the OCIFS utility:

  1. Complete the Prerequisites.
  2. Enable the oci_included repository, which contains the ocifs package. This repository is enabled by default in Oracle Linux platform images.
  3. Install the ocifs package:
    sudo dnf install ocifs

Updating the OCIFS Utility

To update to the latest version of ocifs:

sudo yum update ocifs

OCIFS Utility Details

When you mount an Object Storage bucket with OCIFS, the bucket can be used as a file system, letting you manage the objects in the bucket as regular directories and files.

Although objects in the Object Storage bucket exist in a flat structure, OCIFS simulates a directory structure by using prefix strings that are present in object names that include one or more forward slashes (/). OCIFS is implemented as a FUSE file system, so you don't need root privileges to use the file system.

Note

An OCIFS file system isn't a shared file system. To avoid object data corruption, don't use OCIFS to mount the same Object Storage bucket (or any overlapping bucket subfolder) many times, simultaneously. Also, don't directly change a bucket object (for example, with the Console, CLI or API) while the Object Storage bucket is mounted.

For more information about Object Storage, see Overview of Object Storage.

For information about mounting with OCIFS, see Mounting and Unmounting with OCIFS.

Usage

ocifs [--auth=method] [--config=file] [--region=name] [--cache=path] [--cache-keep] [--cache-reuse] [--cache-fsfree=] [--cache-purge=never|seconds] [--debug=[,...]] [-o ] [-d] [-f] [-s] [--check-bucket bucket-path] [--version] [--mpu-threshold=size] bucket-path directory

OCIFS Utility Options

OCIFS options can be specified with the ocifs command as long options --option[=value] or with the -o option, -o option[=value].

OCIFS Utility Options

Option

Description

For more information, see:

auth=method

Sets the authentication method OCIFS uses when accessing OCI Object Storage:
  • api_key (Default)

  • instance_principal

  • resource_principal

OCIFS Authentication

config=file

Sets the path to the CLI configuration file used for API key-based authentication.

API Key-Based Authentication

region=name

Overrides the region name specified in the CLI configuration file.

API Key-Based Authentication

Cache Options

Defines how the OCIFS cache functions:
  • cache=path: Sets the path to the cache directory.

  • cache-fsfree: Specifies the amount of space to remain free on the file system where the cache directory is located.

  • cache-keep: Doesn't remove the cache directory when the OCIFS file system is unmounted.

  • cache-purge: Specifies the cache purge delay.

  • cache-reuse: Reuses the existing cache directory, even if the directory isn't empty.

OCIFS Cache Options

debug=level

Sets the debug level, or levels, and shows the debugging information in the ocifs command output. Debug levels:

  • all: Debug all operations.

  • cache: Debug cache operations.

  • fops: Debug FUSE operations.

  • oci: Debug OCI requests.

  • other: Debug other, unclassified operations.

For more than one debug level, separate the levels with a comma.

Enabling OCIFS Diagnostics

FUSE Options
The ocifs command accepts the following FUSE options:
  • d: Enables FUSE debug output.

  • f: Runs the ocifs process in the foreground.

  • s: Runs the ocifs process single-threaded instead of multi-threaded.

In addition, FUSE mount options, described in the fuse(8) man page, can be specified with the -o option.

fuse(8) man page

check-bucket bucket-path

Checks that the specified Object Storage bucket is accessible, and that objects in the bucket have names compatible with OCIFS mounting rules.

OCIFS Mounting Restrictions

version

Displays the version of the OCIFS utility.

Updating the OCIFS Utility

mpu-threshold

Defines the file size that, above which, multipart upload is used.

OCIFS Multipart Upload Option

bucket-path

Specifies the Object Storage bucket that you want to mount.

Mounting and Unmounting with OCIFS

Creating a Bucket

directory

Specifies the directory on which to mount the Object Storage bucket.

Mounting and Unmounting with OCIFS

Mounting and Unmounting with OCIFS

With the OCIFS utility, you can specify the Object Storage bucket that you want to mount by its bucket path.

Bucket paths can be either of the following:

  • A bucket name: All objects from the specified Object Storage bucket are accessible as regular files or directories in the mounted directory.

    For example, to mount the Object Storage bucket, bucket-1, on directory mydir:
    $ mkdir ~/mydir
    $ ocifs bucket-1 mydir
  • A bucket subfolder that's an Object Storage bucket name with a subfolder from that bucket: Only objects from the specified bucket subfolder are accessible as regular files or directories in the mounted directory.

    For example, to mount the subfolder marathon/participants from the Object Storage bucket bucket-1 on directory mydir:
    $ ocifs bucket-1/marathon/participants mydir

Mounting an Object Storage Bucket

Prerequisites:

Before using OCIFS to mount an Object Storage bucket, review the following:

To mount an Object Storage bucket:

  • Use the ocifs command:
    ocifs [options] bucket-path directory

    Or:

  • Use the mount(8) command:
    mount -t fuse.ocifs [-o options] bucket-path directory
    Note

    With the mount command, options are specified with the -o option: -o option[=value].

    Note

    The mount command requires the appropriate privileges (such as root).

Examples:

  • Mount the Object Storage bucket bucket-1 on directory mydir using the ocifs command, and keep the OCIFS cache directory after unmounting:

    $ ocifs --cache-keep bucket-1 mydir
  • Mount the Object Storage bucket bucket-1 on directory /mnt using the mount command, and keep the OCIFS cache directory after unmounting:

    # mount -t fuse.ocifs -o cache-keep bucket-1 /mnt 

By default, the Object Storage bucket is mounted using API key-based authentication. See API Key-Based Authentication.

Unmounting an Object Storage Bucket

  • Use the fusermount -u command:
    fusermount -u directory

    Or:

  • Use the umount(8) command:
    umount directory
    Note

    The umount command requires appropriate privileges (such as root).
Examples:
  • Unmount the Object Storage bucket mounted on directory mydir using the fusermount command:

    $ fusermount -u mydir
  • Unmount the Object Storage bucket mounted on directory /mnt using the umount command:

    # umount /mnt
Note

Unmounting an OCIFS file system removes its corresponding cache directory, unless the file system is mounted with the cache-keep option. See cache-keep.

OCIFS Mounting Restrictions

OCIFS has several mounting restrictions to consider before mounting an Object Storage bucket.

Review the following mounting rules and restrictions:

File Types, Attributes and Sizing

OCIFS has the following file type, attribute, and sizing mount restrictions.

File Types
OCIFS supports only regular files and directories. OCIFS does not support symbolic links, physical links, and special files (block, character, fifo, special).
File Attributes
File attributes (permissions, user and group ownership, access/modification/change times) aren't persistent across mounts. All attributes are reset when an OCIFS file system is unmounted and remounted.
By default, permissions are set to 0755 for directories and 0644 for files. Ownership is set with the uid and gid of the owner of the file system. File permissions and ownership can't be changed, unless the OCIFS file system is mounted by the root user. Removing the write permission to a file prevents newly read data from being cached. In this case, OCIFS doesn't have permission to write to the cached file.
File Size
The OCIFS maximum file size is 10 TiB. This is the OCI Object Storage maximum object size.

Naming Rules

Follow the OCIFS mounting naming rules for files, paths, objects, and prefixes.

File Names
OCIFS file name size is limited to the size that's supported by the file system being used for the OCIFS cache. The most common file name size limit for file systems is 255 characters. If an Object Storage bucket has an object with a file name size larger than 255 characters, the object might be visible as a file or directory on the OCIFS file system, but the object is read-only and can't be cached. Because the object isn't cached, the object access time might be slow.
OCIFS read/write access and caching of the object can be restored by renaming the object with a name that's compatible with the file system used for the OCIFS cache. You can use the Oracle Cloud Console to rename an object, or you can rename the object with OCIFS using the mv(1) command.
Path Names
OCIFS supports path names with a maximum of 1023 characters.
Incompatible Object Names
An Object Storage bucket can have objects with names that aren't compatible with Oracle Linux file system naming conventions (For example: //directory or directory////mydirectory). However, these objects aren't accessible nor visible with OCIFS. To be usable with OCIFS, such objects must be renamed with names that are compatible with Oracle Linux file system naming conventions. You can use the Oracle Cloud Console to rename an object.
An object name can have up to 1024 characters, and it can have a corresponding OCIFS file name that's larger than the usual file system file name limit (255 characters). However, access to such objects with OCIFS are limited.
To verify whether an Object Storage bucket has names that aren't compatible with the OCIFS file system, use the --check-bucket option of the ocifs command. See OCIFS Utility Details.
Object and Prefix Name Conflicts
If the same name is used both as an Object Storage object name and as a prefix name, then that name is displayed as a directory entry with OCIFS. The object with the same name isn't accessible through OCIFS. To make the object visible through OCIFS, you must rename the object with a name that's not used as a prefix.
For example, if an Object Storage bucket has objects directory and directory/mydirectory, then directory is used both as an object name and as a prefix name. In that case, OCIFS shows the entry directory as a directory and provides access to all objects starting with prefix directory/ (For example, object directory/mydirectory). However, object directory isn't accessible.
Use the --check-bucket option of the ocifs command to verify if a bucket has conflicting object and prefix names.

Data Synchronization

OCIFS provides different options for data synchronization.

When writing to an OCIFS file, data is synchronized (data is effectively written back to the corresponding Object Storage object) when the OCIFS file is closed. Any data synchronization error is reported by the close(2) system call. For information about dealing with error returns from close(), see the NOTES section of the close(2) man page. You can also request data to be explicitly synchronized using fsync(2) or fdatasync(2).

Or, you can make write operations immediately synchronize data either for a specific file (by opening the file with the O_SYNC or O_DSYNC flag) or for all files (by mounting the OCIFS file system with the sync option (-o sync).
Note

Using the O_SYNC or O_DSYNC open flags, or the sync mount option can negatively impact performance.

OCIFS Authentication

Use the API key-based (default), instance principal, or resource principal authentication methods with the OCIFS utility.

When mounting an Object Storage bucket with the OCIFS utility, you can use the auth=method option with the ocifs command to specify which authentication method to use for accessing and authenticating Object Storage.

For more information, see:

API Key-Based Authentication

Specify and configure the API key-based authentication method to mount Object Storage buckets with OCIFS.

API key-based authentication is the default authentication method for OCIFS, and can be used from an OCI compute instance or from any other system. This authentication method requires a local OCI SDK and a CLI configuration file. You can specify the configuration file by using the config=file option with the ocifs command. By default, the configuration file is ~/.oci/config.

For example, if you mount the Object Storage bucket, bucket-1, on the mydir directory, and you want to use API key-based authentication with the ~/my_config CLI configuration file:
$ ocifs --auth=api_key --config=~/my_config bucket-1 mydir

Before mounting an Object Storage bucket using the API key-based authentication method, ensure that the CLI configuration file (~/.oci/config) has a [DEFAULT] profile, and the entries specified in the following table.

Configuration File Entries for API Key-Based Authentication

Entry

Description

Example

user

OCID of the user who is accessing the Oracle Cloud Infrastructure

user=ocid1.user.oc1..unique_ID

tenancy

OCID of your tenancy

tenancy=ocid1.tenancy.oc1..unique_ID

region

An Oracle Cloud Infrastructure region

region=regional_area_name

key_file

Full path and file name to the private key

key_file=~/.oci/oci_api_key.pem

fingerprint

Fingerprint for the public key used for this user

fingerprint=00:11:22:33:44:55:66:77:88:99:aa:bb:cc:dd:ee:ff

For more information, see SDK and CLI Configuration File.

Instance Principal Authentication

Instead of the default API key-based authentication method, you can use the instance principal authentication method with the OCIFS utility.

Use the instance principal authentication method for OCIFS only from an authorized Oracle Cloud Infrastructure instance. This authentication method lets specified instances access Object Storage without a CLI configuration file. The instance then automatically discovers information to authenticate.

Instance principal authentication must be used to mount an Object Storage bucket on a compute instance that has been configured as an instance principal.

For example, to mount an Object Storage bucket, bucket-1, on the mydir directory using instance principal authentication:

$ ocifs --auth=instance_principal bucket-1 mydir

To use instance principal authentication, a compute instance must be part of a dynamic group, and a policy must allow the dynamic group to manage Object Storage. You can use the manage object-family verb and resource type to define a policy that allows managing of Object Storage objects. For more information about dynamic groups, see Managing Dynamic Groups.

For example, to allow a compute instance (with instance id ocid1.instance.OCID) to use instance principal authentication:

  1. Create a dynamic group dgroup-ocifs with the following rule that matches the compute instance:

    All {instance.id = 'ocid1.instance.OCID'}
  2. Create a policy that allows that dynamic group, dgroup-ocifs, to manage Object Storage:

    Allow dynamic-group dgroup-ocifs to manage object-family in compartment id ocid1.tenancy.tenancy_ID

For more information, see Calling Services from an Instance.

Resource Principal Authentication

Instead of the default API key-based authentication method, you can use the resource principal authentication method with OCIFS.

Use the resource principal authentication method with OCIFS only from an authorized Oracle Cloud Infrastructure (OCI) function. This authentication method lets the specified function access another resource, or Object Storage, without a CLI configuration file. The OCI function then automatically discovers information to authenticate.

To use resource authentication, an OCI function must be part of a dynamic group, and a policy must allow the dynamic group to access that resource, or Object Storage. For more information about dynamic groups, see Managing Dynamic Groups.

For information about OCI functions accessing other OCI resources, see Accessing Other Oracle Cloud Infrastructure Resources.

To use the resource authentication method with OCIFS, enter --auth=resource_principal when mounting a bucket with the ocifs command.

For example, to mount an Object Storage bucket, bucket-1, on the mydir directory using resource principal authentication:

$ ocifs --auth=resource_principal bucket-1 mydir

Enabling OCIFS Diagnostics

To enable OCIFS diagnostics, use the ocifs command with the -f option to run the OCIFS process in the foreground, and specify debug levels with the --debug option.

For example, to display diagnostic information on all OCIFS operations for the Object Storage bucket bucket-1, mounted on the mydir directory:

$ ocifs -f --debug=all bucket-1 mydir

Or, you can use the -d option with the ocifs command to enable FUSE debugging on all operations.

For example:

$ ocifs -d --debug=all bucket-1 mydir

OCIFS Cache Options

Use OCIFS cache options to specify how the OCIFS cache functions.

The OCIFS utility caches objects of a mounted Object Storage bucket path in a local directory (by default: ~/.ocifs/bucket/).

If the storage device where the cache directory is located gets full, then some OCIFS operations might fail.

You can specify how the OCIFS cache is used and functions with the following ocifs cache options:

cache

You can change the OCIFS cache directory with the ocifs command cache=path option.

By default, OCIFS caches objects in ~/.ocifs/. You can change this cache directory using the cache=path option. This option sets the cache directory to the specified path. The path can be an absolute path or relative path to the user's home directory.

For example, to mount the Object Storage bucket bucket-1 on the mydir directory, and use /var/tmp/ocifs-cache as the OCIFS cache directory:
$ ocifs --cache=/var/tmp/ocifs-cache bucket-1 mydir

cache-fsfree

You can prevent the OCIFS utility from accidentally filling up space on the cache file system with the ocifs command cache-fsfree option.

Use the cache-fsfree option to specify the amount of space that must remain free on the cache file system. If the free space is greater than the amount specified, then OCIFS tries to cache data. Otherwise, OCIFS doesn't perform caching. If OCIFS is unable to cache data, then some OCIFS operations can fail, such as write operations.

Note

The cache-fsfree option doesn't change the way the attributes for OCIFS entries (such as the size) are cached.

The cache directory size values are listed in the following table.

Cache Directory Size Values

Value

Description

Example

0

Don't check for free space on the cache file system. With this option, OCIFS tries to cache data, no matter how much free space is available on the cache file system.

$ ocifs --cache-fsfree=0 bucket-1 mydir

percent%

This value specifies the percentage of space (between 0% and 100%) that must remain free on the cache file system.

If the percentage is 100% then no data is cached.

$ ocifs --cache-fsfree=20% bucket-1 mydir

number[K|M|G|T]

This value specifies the amount of space in Kilobytes (K), Megabytes (M), Gigabytes (G) or Terabytes (T) that must remain free on the cache file system.

$ ocifs --cache-fsfree=1G bucket-1 mydir

number[K|M|G|T]i

This value specifies the amount of space in Kibibytes (Ki), Mebibytes (Mi), Gibibytes (Gi) or Tebibytes (Ti) that must remain free on the cache file system.

$ ocifs --cache-fsfree=1Gi bucket-1 mydir

Notes:

  • The default setting is cache-fsfree=5%.
  • Using the option 0, 0%, 0[K|M|G|T], or 0[K|M|G|T]i has the same effect.
  • Using the option 100% or an amount of space larger, or equal than the size of the file system, has the same effect.

cache-keep

You can retain the OCIFS cache with the ocifs command cache-keep option, even if the OCIFS file system is unmounted.

Unmounting the OCIFS file system removes the OCIFS cache directory, unless you mount the file system with the cache-keep option.

For example, to mount an Object Storage bucket, bucket-1, on the mydir directory, and have the cache preserved after unmounting:

$ ocifs --cache-keep bucket-1 mydir

Notes:

  • If you use the cache-keep option, you might be unable to mount the file system again unless you then use the cache-reuse option.
  • Keeping data in cache after unmounting the file system can improve performance the next time you mount the file system. However, OCIFS doesn't check if there are any discrepancies between data in cache and the actual data in the Object Storage bucket.

cache-purge

The OCIFS utility cache-purge option lets you remove data from the OCIFS cache for files that have been closed for a specified amount of time.

By default, the OCIFS cache is never purged. The cache data is kept in the cache directory as long as the file system is mounted. The cache directory is only removed when the OCIFS file system is unmounted.

Using the cache-purge option, you can specify the amount of time data remains in cache after a file is closed. You can use this to preserve disk space, especially if files aren't often accessed. This is useful, for example, if you write new files and don't access the files again.

The OCIFS cache purge delay can be specified as listed in the following table:

OCIFS Cache Purge Delay Values

Value

Description

Example

never

Never purge the OCIFS cache. With this value, cached data is preserved as long as the OCIFS file system is mounted. (Default)

$ ocifs --cache-purge=never bucket-1 mydir

delay-time

The purge delay, in seconds, before purging cached data. If the purge delay is set to 0, then cached data for a file is purged immediately after the file is closed.

$ ocifs --cache-purge=30 bucket-1 mydir

Note

Setting an OCIFS cache purge delay, in particular with a small value, can negatively impact performance. This is because data is eventually removed from the cache when a file isn't used. If a file is reopened later and read, performance can be slow as the file data isn't in the cache.

cache-reuse

The OCIFS utility cache-reuse option lets an OCIFS file system be mounted, even if there is an existing, nonempty cache directory.

By default, OCIFS prevents mounting a file system if it has an nonempty cache directory. With the cache-reuse option, you can remount the file system and reuse the existing cache directory, if present.

For example, to mount the Object Storage bucket, bucket-1, on the mydir directory, and reuse the existing cache directory:

$ ocifs --cache-reuse bucket-1 mydir

Notes:

  • You can have a nonempty cache directory if the file system was previously mounted with the cache-keep option, or if the ocifs process was killed or has crashed.
  • Reusing an existing cache can improve performance, as the OCIFS utility can access data already in the cache more quickly. However, using the cache-reuse option isn't recommended because OCIFS doesn't check for discrepancies between data in cache and the actual data in the Object Storage bucket.

OCIFS Multipart Upload Option

OCIFS uses Oracle Cloud Interface (OCI) Object Storage multipart upload to upload large objects to OCI Object Storage.

When OCIFS uses multipart upload to upload an object, it fairly divides the object into parts with a maximum part size of 50 GiB. Parts are then uploaded by a pool of threads.

You can specify how OCIFS uses multipart upload with the OCIFS mpu-threshold command option. For more information, see mpu-threshold.

mpu-threshold

OCIFS lets you determine when it uses mulipart upload to upload large files to Object Storage. With the OCIFS mpu-threshold=size option, you can specify the file size that, above which, multipart upload starts.

The default multipart upload size threshold value is 100 Mebibytes (recommended).

The following table lists the supported mpu-threshold=size option values.

OCIFS mpu-threshold Values

Value

Description

Example

0

Always use multipart upload (not recommended).

$ ocifs --mpu-threshold=0 bucket-1 mydir

number[KMGT]

The size in Kilobytes (K), Megabytes (M), Gigabytes (G) or Terabytes (T) above which multipart upload is used.

$ ocifs --mpu-threshold=150M bucket-1 mydir

number[KMGT]i

The size in Kibibytes (Ki), Mebibytes (Mi), Gibibytes (Gi) or Tebibytes (Ti) above which multipart upload is used.

$ ocifs --mpu-threshold=150Mi bucket-1 mydir