Oracle Cloud Infrastructure Documentation

Mounting and Accessing a Lustre File System

To access a File Storage with Lustre file system, you need to mount it from a Compute instance or OKE node. On the instance from which you want to mount the file system, you need to install a Lustre client. First, build the Lustre client package on a system, then install the client package on all instances that need to mount a Lustre file system.

Clients

A Lustre client running on a Compute instance is required to access a Lustre file system. Build and install a client that's compatible with the OS of the instances mounting the file system. If you need help with Lustre clients, contact support.

Building a Lustre client for Oracle Linux 8 or 9

Build a Lustre client on one system and then install the client package on each instance that mounts a Lustre file system.

  1. To use a Lustre client with Oracle Linux, the kernel must be changed to Red Hat Compatible Kernel (RHCK). Switching kernel versions is described in Changing the default kernel in Oracle Linux.
  2. Ensure that the system is running the RHCK kernel after reboot.

  3. Install required packages using the following commands:

    release=$(cat /etc/os-release | grep ^VERSION= |cut -f2 -d\" |cut -f1 -d.)
    sudo dnf config-manager --set-enabled ol${release}_codeready_builder
    sudo dnf config-manager --enable ol${release}_developer_EPEL
    sudo yum-config-manager --enable ol${release}_developer
    sudo yum install git libtool patch pkgconfig libnl3-devel.x86_64 libblkid-devel libuuid-devel rpm-build kernel-rpm-macros kernel-devel kernel-abi-whitelists libmount libmount-devel libyaml-devel

  4. Clone the Lustre client source repo and checkout the client using the following commands:

    git clone git://git.whamcloud.com/fs/lustre-release.git lustre-client
    cd lustre-client/
    git checkout tags/2.15.5

  5. Build the client and RPMs. All Lustre RPMs in the current directory are built using the following commands:

    sudo sh autogen.sh
./configure --enable-client
sudo make
sudo make rpms
sudo make dkms-rpm

  6. Inspect the RPMs:

    ls -l *.rpm

  7. If you plan to install the Lustre client on the same instance you built it, run the following commands:

    sudo make install
    sudo depmod
    sudo modprobe lustre

Proceed to Installing a Lustre client for Oracle Linux 8 or 9.

Installing a Lustre client for Oracle Linux 8 or 9

These steps assume that you already built a Lustre client for Oracle Linux.

In this example, the Lustre Dynamic Kernel Module Support (DKMS) client package is used for its flexibility across different kernel versions. The DKMS RPMs you built work with both Oracle Linux 8 and Oracle Linux 9. However, if you're using the DKMS client, Lustre modules must be built when it's installed or the kernel gets upgraded. You can choose the RPMs that you want to install, such as the non-DKMS kmod-lustre-client package for the Linux modules.

  1. Create a directory on the instance where you want to install the Lustre client. For example:

    mkdir lustre-rpms

  2. Copy the Lustre RPMs. In this example, we use the DKMS RPMs, such as:

    lustre-client-2.15.5-1.el8.x86_64.rpm
lustre-client-dkms-2.15.5-1.el8.noarch.rpm
  3. To use a Lustre client with Oracle Linux, the kernel must be changed to Red Hat Compatible Kernel (RHCK). Switching kernel versions is described in Changing the default kernel in Oracle Linux.
  4. Ensure that the system is running the RHCK kernel after reboot.

  5. Enable required Linux repository for dependencies, and install dependencies using the following commands:

    release=$(cat /etc/os-release | grep ^VERSION= |cut -f2 -d\" |cut -f1 -d.)
    sudo dnf config-manager --set-enabled ol${release}_codeready_builder
    sudo dnf config-manager --enable ol${release}_developer_EPEL
    sudo yum install dkms libmount-devel libyaml-devel

  6. Install the client RPMs using the following command:

    sudo yum localinstall -y ./*.rpm

  7. Test the modules using the following commands:

    sudo modprobe lustre
    sudo lsmod |grep lustre

  8. The Lustre client is ready. Reboot the system.

Building a Lustre client for Ubuntu 22.04

Build a Lustre client on one system and then install the client package on each instance that mounts a Lustre file system.

  1. Lustre 2.15.5 isn't compatible with Linux kernel 6. The client needs a kernel version of 5.15.x. Check the OS and kernel versions using the following commands:

    cat /etc/os-release
    uname -r

    In this example, the compatible kernel version used is 5.15.0-1040-oracle.

  2. If a different kernel version is installed, install the required kernel version and kernel headers using the following commands, which use the example version of 5.15.0-1040-oracle:

    sudo apt-get install linux-image-5.15.0-1040-oracle
    sudo apt-get install linux-headers-5.15.0-1040-oracle
    sudo apt-get install linux-modules-5.15.0-1040-oracle

  3. Update grub using the following command:

    sudo update-grub

  4. Inspect the /boot/grub/grub.cfg to get the entry for the kernel version and update the entry GRUB_DEFAULT in the file /etc/default/grub as follows:

    GRUB_DEFAULT='Advanced options for Ubuntu>Ubuntu, with Linux 5.15.0-1040-oracle'

  5. Update grub using the following command:

    sudo update-grub
  6. Reboot the client using sudo reboot. After the reboot, ensure that the correct kernel version is running.

  7. Clone the Lustre client source repo and checkout the client using the following commands:

    git clone git://git.whamcloud.com/fs/lustre-release.git lustre-client
    cd lustre-client/
    git checkout tags/2.15.5

  8. Install common packages using the following command:

    sudo apt-get install -y libreadline-dev libpython3-dev libkrb5-dev libkeyutils-dev flex bison libmount-dev quilt swig libtool make git libnl-3-dev libnl-genl-3-dev libnl-3-dev pkg-config libhwloc-dev libnl-genl-3-dev libyaml-dev libtool libyaml-dev ed libreadline-dev dpatch libsnmp-dev mpi-default-dev libncurses5-dev libncurses-dev bison flex gnupg libelf-dev gcc libssl-dev bc wget bzip2 build-essential udev kmod cpio module-assistant module-assistant debhelper dpatch libsnmp-dev mpi-default-dev libssl-dev

  9. Configure the client:

    cd lustre-client
    sudo sh autogen.sh
    ./configure --enable-client

  10. Edit the Debian control files to remove dependencies with latest linux-image and linux-headers packages. Without removing these dependencies, building the packages will fail. Edit the control files such as:

    vi debian/control*

    Remove all the references in the two files, such as:

    linux-headers-generic | linux-headers-amd64 | linux-headers-arm64 , linux-image | linux-image-amd64 | linux-image-arm64, linux-headers-generic | linux-headers-amd64

  11. Build the packages using the following commands:

    sudo make debs
    sudo make dkms-debs

  12. Inspect the packages:

    ls -l debs/*.deb

  13. If you plan to install the Lustre client on the same instance you built it, run the following commands:

    sudo make install
    sudo depmod
    sudo modprobe lustre

Proceed to Installing a Lustre client for Ubuntu 22.04.

Installing a Lustre client for Ubuntu 22.04

These steps assume that you already built a Lustre client for Ubuntu.

  1. Lustre 2.15.5 isn't compatible with Linux kernel 6. The client needs a kernel version of 5.15.x. Check the OS and kernel versions using the following commands:

    cat /etc/os-release
    uname -r

    In this example, the compatible kernel version used is 5.15.0-1040-oracle.

  2. If a different kernel version is installed, install the required kernel version and kernel headers using the following commands, which use the example version of 5.15.0-1040-oracle:

    sudo apt-get install linux-image-5.15.0-1040-oracle
    sudo apt-get install linux-headers-5.15.0-1040-oracle
    sudo apt-get install linux-modules-5.15.0-1040-oracle

  3. Update grub using the following command:

    sudo update-grub

  4. Update the entry GRUB_DEFAULT in the file /etc/default/grub as follows:

    GRUB_DEFAULT='Advanced options for Ubuntu>Ubuntu, with Linux 5.15.0-1040-oracle'

  5. Update grub using the following command:

    sudo update-grub
  6. Reboot the client using sudo reboot. After the reboot, ensure that the correct kernel version is running.

  7. Copy the package files that you built to the instance where you want to install the Lustre client. In the following example debs directory, the DKMS kernel module package was selected for its flexibility, but you may choose non-DKMS module RPMs if you prefer:

    rw-r--r-- 1 ubuntu ubuntu 17058172 Apr  2 02:26 lustre-client-modules-dkms_2.15.5-1_amd64.deb
-rw-r--r-- 1 ubuntu ubuntu   678790 Apr  2 02:26 lustre-client-utils_2.15.5-1_amd64.deb

  8. Install the packages using the following commands:

    sudo apt-get update
    sudo apt install --fix-broken ./*.deb

  9. Test the modules using the following commands:

    sudo modprobe lustre
    sudo lsmod |grep lustre

  10. The Lustre client is ready. Reboot the system.

Mount Commands

From an instance with Lustre client modules installed, you can mount a file system.

Getting the mount command

Use the following steps to get the mount command for a particular file system:

  1. On the Lustre file systems list page, select the file system that you want to work with. If you need help finding the list page, see Listing File Systems.
  2. On the file system's details page, under Lustre properties, find Mount command.
  3. Select Copy to copy the mount command to the clipboard.

Mounting a file system

From the instance, run the mount command copied to the clipboard in the previous steps. For example:

mount -t lustre 10.0.0.2@tcp:/lustrefs /mnt/mymountpoint