Oracle Cloud Infrastructure Documentation

Installing a Cluster with Agent-based Installer Using Terraform

Learn how to provision the OCI infrastructure for the Red Hat OpenShift Container Platform using a Terraform script with the Agent-based installer.

Use the Red Hat OpenShift plugin to automatically provision infrastructure with Terraform. The latest version of the Terraform stack is available on the Stack information page within the Red Hat OpenShift plugin.

You can access the earlier versions of the Terraform from the OpenShift on OCI Releases page on GitHub. Download your required version of the create-cluster.zip file from the Assets section.

If you are creating an autoscaling stack, use the create-cluster zip you downloaded in Prerequisites for Autoscaling.

Important

  • Run the create-resource-attribution-tags stack before running the create-cluster stack to avoid installation failure. See the Prerequisites topic for more information.
  • The create-resource-attribution-tags stack only needs to be run once. If the tag namespace and defined-tags already exist, you can skip this step for future installations.
  • A tagging controller runs in the background to ensure the required OpenShift resource attribution tags are present on cluster resources during installation and ongoing reconciliation.
  • OpenShift requires manage permissions to perform operations on instances, volumes, and networking resources. Deploy OpenShift in a dedicated compartment to avoid conflicts with other applications that might be running in the same compartment.

See Terraform Defined Resources for OpenShift for a list of the resources created by the stack.

  1. Open the navigation menu  and select Developer Services, then select Red Hat OpenShift.
    • The latest version of the Terraform stack is automatically uploaded for you.
    • The Create in compartment and Terraform version fields are prepopulated.
    • Enter an optional Name and Description.
  2. Select Next.
  3. On the Configure variables page, review and configure the required variables for the infrastructure resources that the stack creates when you run the Apply job for the first time.
  4. In the OpenShift Cluster Configuration section:
    • Change the Installation Method to Agent-based.
    • Configure the compute and control-plane counts. These values are used when generating the install-config.yaml file for the Agent-based Installer.
    • Clear the Create OpenShift Image and Instances checkbox.

    OpenShift Cluster Configuration

    Field Description
    Tenancy OCID

    The OCID of the current tenancy.

    Default value: Current tenancy
    Compartment

    The OCID of the compartment where you want to create the OpenShift cluster.

    Default value: Current compartment
    Cluster Name

    The OpenShift cluster name.

    Note: Use the same value for cluster name that you specified in your agent-config.yaml and install-config.yaml files and ensure that it's DNS compatible.
    Installation Method

    The installation method you want to use for installing the cluster.

    Note: For Agent-based installation, select Agent-based.

    Default value: Assisted Installer
    Create OpenShift Image and Instances

    Enables the creation of OpenShift image and instances.

    Default value: True
    OpenShift Image Source URI

    The pre-authenticated Request URL created in the previous task, Uploading Red Hat ISO Image to Object Storage.

    Note: This field is visible only when the Create OpenShift Image and Instances checkbox is selected later.
  5. In the OpenShift Resource Attribution Tags section, review and configure the following fields:

    OpenShift Resource Attribution Tags
    Field Description
    Tag Namespace Compartment For OpenShift Resource Attribution Tags

    The tag namespace compartment for OpenShift resource attribution tags.

    Note: Ensure that the specified tag exists before applying the Terraform stack. The compartment where the tag namespace for resource tagging is created defaults to the current compartment. For OpenShift Attribution on OCI resources, the tag namespace and defined tags are set to openshift-tags and openshift-resource. If the openshift-tags namespace already exists, ensure it's correctly defined and applied. For example: defined-tags" - {"openshift-tags"- {"openshift-resource" - "openshift-resource-infra"} }
  6. In the Agent-based Installation Advanced Configurations section, review and configure the following fields:
    Note

    This section appears only when the Installation Method is set to Agent-Based Installer.

    Agent-based Installation Advanced Configurations
    Field Description
    Rendezvous IP

    The IP used to bootstrap the cluster using the Agent-based Installer. Must be in private_ocp subnet.
    Is Disconnected Agent-based Installation

    Indicates whether the cluster is being installed in a disconnected (air-gapped) environment. Checking this box also enables the creation of a webserver, which can be used to host the agent.x86_64-rootfs.img file and facilitate the Agent-based installation.

    Default value: False
    Set OpenShift Installer Version

    The OpenShift installer version you want to use. Specify a specific supported version if you don't want the latest version. For example, 4.19.1.

    Default value: False
    OpenShift Installer Version

    The version of OpenShift installer. You can find a list of published versions on the OpenShift Client Downloads page.

    Default value: Latest
    Public SSH

    The public SSH key for the webserver to use.

    Note: This key is also added to the install-config.yaml file and is required to provide access to the Openshift instances.
    Red Hat Pull Secret

    The pull secret for authentication when downloading container images for OpenShift Container Platform components and services from sources such as Quay.io. For more information, see Install OpenShift Container Platform 4.
    Object Storage Namespace The OCI Storage namespace for the tenancy.

    You can find this value in the Bucket Details page for your bucket. For more information, see Object Storage Namespaces.
    Object Storage Bucket The OCI Object Storage bucket where the OpenShift installation files are stored.
    Webserver Private IP The private IP of the server where you want to upload the rootfs image. This parameter is required only for disconnected environments. This IP must be included in the bootArtifactsBaseURL value in the agent-config.yaml file.
    Webserver Shape The Compute instance shape of the the webserver instance. For more information, see Supported Shapes.
    Webserver Image

    The source_id of the image to use for the webserver instance. If an image isn't preselected, select from a list of available Oracle Linux images.
    Webserver OCPU

    The number of OCPUs available for the webserver shape instance.

    Default value: 2
    Webserver Memory

    The amount of memory available, in gigabytes, for the webserver shape instance.

    Default value: 8
    Config Proxy Settings Provide additional information about the proxy if hosts are behind a firewall that requires the use of a proxy.
    HTTP Proxy URL Specify the HTTP proxy URL. For example, http://my.proxy.server.
    HTTPS Proxy URL Specify the HTTPS proxy URL. For example, https://my.proxy.server.
    No Proxy Domains Specify the No Proxy Domains. For example, localhost,127.0.0.1,{basedomain}.
  7. In the Network Configuration section, review and configure the following fields:

    Network Configuration
    Field Description
    Create Public DNS

    Create a public DNS zone based on the Base domain specified in the Zone DNS field.

    Note: If you don't want to create a public DNS, create a private DNS zone unless you use your own DNS solution. To resolve cluster hostnames without DNS, add entries to /etc/hosts that map cluster hostnames to the IP address of the api_apps Load Balancer using the etc_hosts_entry output.

    Default value: True
    Create Private DNS

    Creates a private DNS zone based on the Base domain, specified in the Zone DNS field, to support hostname resolution within the VCN.

    Note: The private DNS zone contains the same records as the public DNS zone. If using an unregistered Base domain, we recommend to create a private DNS zone, otherwise, you might need alternative methods to resolve cluster hostnames. For more information, see Private DNS.

    Default Value: False
    Enable Public API Load Balancer

    Creates a Load Balancer for the OpenShift API endpoint in the public subnet with a public IP address for external access.

    Note: If unselected, the Load Balancer is created in a private subnet with limited access within the VCN or connected private networks. In on-premises environments (for example, C3/PCA), public IPs might be internal (RFC 1918). Public access is helpful for remote management, automation, and CI/CD. Contact your network administrator if needed.

    Default Value: False
    Enable Public Apps Load Balancer

    Creates a Load Balancer for OpenShift applications in the public subnet with a public IP address for external access to workloads. For example, console-openshift-console.apps.mycluster.mydomain.com or console-openshift-console.apps.devcluster.openshift.com

    Note: If unselected, the Load Balancer is created in a private subnet, limiting access to within the VCN or over a VPN/private network. Public access is useful for internet-facing apps, customer services, or multitenant workloads. In on-premises setups (for example, C3/PCA), public IPs might be internal (RFC 1918). Contact your network team to ensure proper exposure.

    Default value: True
    Zone DNS The base domain for your cluster's DNS records (for example, devcluster.openshift.com), which is used to create public or private (or both) DNS Zones. This value must match what is specified for baseDomain in the install-config.yaml file.
    Use Existing Networking Infrastructure

    Select to reuse an existing VCN and subnets instead of creating new networking resources. Before selecting this option, complete Prerequisites for Using an Existing VCN.

    Default value: False
    VCN Compartment

    The compartment that contains the existing VCN. This field is displayed only when Use Existing Networking Infrastructure is selected.

    Default value: Current compartment
    Existing VCN

    The OCID of the existing VCN. This field is displayed only when Use Existing Networking Infrastructure is selected.
    Subnet Compartment

    The compartment that contains the existing subnets. This field is displayed only when Use Existing Networking Infrastructure is selected.

    Default value: Current compartment
    Existing Private Subnet for OCP

    The OCID of the existing private subnet used by the OpenShift control-plane and VM compute nodes.
    Existing Private Subnet for Bare Metal

    The OCID of the existing private subnet used by bare-metal compute nodes. This subnet must be different from the private OpenShift subnet.
    Existing Public Subnet

    The OCID of the existing public subnet used by public-facing cluster resources.
    VCN DNS Label

    A DNS label for the VCN, used with the VNIC's hostname and subnet's DNS label to form a Fully Qualified Domain Name (FQDN) for each VNIC within this subnet. (for example, bminstance1.subnet123.vcn1.oraclevcn.com).

    Default value: openshiftvcn
    VCN CIDR

    The IPv4 CIDR blocks for the VCN of your OpenShift Cluster.

    Default value: 10.0.0.0/16
    Public Subnet CIDR

    The IPv4 CIDR blocks for the public subnet of your OpenShift Cluster.

    Default value: 10.0.0.0/20
    Private Subnet CIDR for OCP

    The IPv4 CIDR blocks for the private subnet of your OpenShift Cluster.

    Default value: 10.0.16.0/20
    Reserved Private Subnet CIDR for Bare Metal

    The IPv4 CIDR blocks for the private subnet of OpenShift Bare metal Clusters.

    Default value: 10.0.32.0/20

    Load Balancer Maximum Bandwidth

    Bandwidth (Mbps) that decides the maximum bandwidth (ingress plus egress) that the Load Balancer can achieve. The values must be between minimumBandwidthInMbps and 8000.

    Default value: 500
    Load Balancer Minimum Bandwidth

    Bandwidth (Mbps that decides the total pre-provisioned bandwidth (ingress plus egress). The values must be between 10 and the maximumBandwidthInMbps.

    Default value: 10
  8. (Optional) In the Advanced Configuration section, review and configure the following fields:

    Advanced Configurations
    Field Description
    OCI CCM and CSI Driver Version

    The OCI CCM and CSI driver version to deploy. For more information see the list of driver versions.

    Default value: v1.34.0
    Use Existing Instance Role Tags

    Indicates whether to reuse existing instance role tag namespace and defined tags when tagging OCI resources. By default, a new set of instance role tagging resources are created and destroyed with the rest of the cluster resources.

    If required, create instance role tag resources separately that can be reused with the Terraform stack from the create-instance-role-tags page in the oci-openshift GitHub repo. The existing instance role tagging resources aren't destroyed when the cluster is deleted.

    Default Value: False
    (Optional) Instance Role Tag Namespace Name

    The name of the instance role tag namespace to reuse.

    Note: This field is visible only when the Use Existing Instance Role Tags checkbox is selected.

    Default value: openshift-{cluster_name}
    Instance Role Tag Namespace Compartment OCID

    The compartment containing existing instance role tag namespace.

    Note: This field is visible only when the Use Existing Instance Role Tags checkbox is selected.

    Default value: Current compartment
    Enable Oracle Cloud Agent (beta)

    Select to enable Oracle Cloud Agent in the cluster. Before selecting this option, accept the Marketplace listing terms and export the agent container image to an OCIR repository in the tenancy.

    Default value: False
    Cloud Agent Container Repository Name

    The name of the OCIR repository that contains the Oracle Cloud Agent container image. This field is displayed only when Enable Oracle Cloud Agent (beta) is selected.

    Default value: openshift-oca
    Region where container image is exported

    Enter a different region only when the Oracle Cloud Agent container image was exported to a region other than the region where the stack is deployed. This field is displayed only when Enable Oracle Cloud Agent (beta) is selected.
  9. Select Next.
  10. On the Review page, review the stack information and variables.
  11. Select Create to create the stack. The Console redirects to the Resource Manager stack details page for the new stack.
  12. On the stack details page, select Apply to create an apply job and provision the infrastructure for the cluster. After running an apply job, get the job's details to check its status. Succeeded (SUCCEEDED) indicates that the job has completed. When it completes, all the OpenShift resources except the ISO image and the Compute instances are created.
  13. On the Stacks page in Resource Manager, select the name of the stack to see its details. If the list view of jobs isn't displayed, select Jobs under the Resources section to see the list of jobs.
  14. Select the job for the stack creation. The Job details page is displayed in the Console.
  15. Select Outputs under the Resources section to see the list of outputs for the job.
  16. (Optional) Select show to view and verify the contents of the relevant Agent-based outputs: agent_config, install_config, dynamic_custom_manifest.
  17. Select copy to copy the contents of Agent-based outputs.
    Note

    • We recommend that you don't manually select and copy the text, as this can cause problems with indentation when you paste this output in the following step.
    • For disconnected installations with a webserver, the output files are already available on the webserver ready to use. They're also available in your Object Storage bucket.
  18. Using a text or code editor, save the copied outputs to the following files: agent-config.yaml, install-config.yaml, dynamic-custom-manifest.yaml file. Or, you can download them from the Object Storage bucket.
  19. Create configuration files and a bootable ISO image for installing your cluster. See Creating configuration files for installing a cluster on OCI (Red Hat documentation) for instructions on:
    • Preparing and customizing your installation files.
    • Downloading an Agent-based Installer.
    • Generating the ISO image.
    • (For disconnected environments) Moving rootfs image to /var/www/html on your webserver.
    • Saving the kubeconfig and kubeadmin-password from the auth folder.
    Return to this documentation after you create the configuration files and ISO image, then continue the installation.
  20. Upload the discovery ISO image file to a bucket in OCI Object Storage. For Putting Data into Object Storage if you need instructions.
  21. Navigate to the Resource Manager service and access the stack details page for the stack you created to install OpenShift.
  22. Under Resources, select Variables.
  23. Select Edit variables.
  24. In the OpenShift Cluster Configuration section:
    • Select the Create OpenShift Image and Instances checkbox.
    • Paste the pre-authenticated request string into the OpenShift Image Source URI field.
  25. Review the values in the Control Plane Node Configuration, Autoscaling Configuration (if available), and Compute Node Configuration sections.
    Note

    Ensure that the Control Plane Node Count and Compute Node Count values match the values in the install-config.yaml file when the image was created.

    Control Plane Node Configuration

    Note: This section is visible only when the Create OpenShift Image and Instances checkbox is selected.

    Field Description
    Control Plane Shape

    The compute instance shape of the control plane nodes. For more information, see Compute Shapes.

    Default value: VM.Standard.E5.Flex
    (Optional) Control Plane Capacity Reservation OCID

    The OCID of the capacity reservation used for the control-plane nodes.
    Control Plane Node Count

    The number of control plane nodes in the cluster.

    Default value: 3
    Control Plane Node OCPU

    The number of OCPUs available each control plane node shape.

    Default value: 4
    Control Plane Node Memory

    The amount of memory available, in gigabytes (GB), for each control plane node shape.

    Default value: 24
    Control Plane Node Boot Volume

    The boot volume size (GB), of each control plane node.

    Default value: 1024
    Control Plane Node VPU

    The number of Volume Performance Units (VPUs) applied to the volume, per 1 GB, of each control plane node.

    Default value: 100
    Distribute Control Plane Instances Across ADs

    Enables automatic distribution of control-plane instances across Availability domains (ADs) in a round-robin sequence, starting from the selected AD. If unselected, all nodes are created in the selected starting AD.

    Default value: True
    (Optional) Starting AD

    Availability domain (AD) used for initial node placement. Additional nodes are automatically distributed across ADs in a round-robin sequence, starting from the selected AD.
    Distribute Control Plane Instances Across FDs

    Distributes control plane instances across fault domains in a round-robin sequence. If not selected, the OCI Compute service distributes them based on shape availability and other criteria.

    Default value: True
    (Optional) Starting FD

    The fault domain used for initial control-plane node placement. Additional nodes are distributed across fault domains in round-robin order, beginning with the selected fault domain.

    Autoscaling Configuration

    Note: This section is visible only when you select the Enable OCI OpenShift Autoscaler checkbox. If you select this option, the following Compute Node Configuration is not available.

    Field Description
    Autoscaler Node Shape

    The instance shape of the Autoscaler compute nodes. For more information, see Compute Shapes.

    Default value: VM.Standard.E5.Flex
    Autoscaler Image Source URI The object storage URI to your Autoscaler image. You created this image as a prerequisite: see Prerequisites for Autoscaling
    Minimum Autoscaler Node Count

    The minimum number of compute nodes in the cluster.

    Default value: 0
    Maximum Autoscaler Node Count

    The maximum number of compute nodes in the cluster.

    Default value: 5
    Autoscaler Pool Identifier Optionally, enter an identifier of up to five lowercase letters, numbers, or hyphens. This identifier is appended to the CAPI cluster name when the autoscaler creates node pool resources.
    Autoscaler Node OCPU

    The number of OCPUs available for each autoscaled compute node shape. This value is only used for VMs, and is ignored for Bare Metal instances.

    Default value: 4
    Autoscaler Node Memory

    The amount of memory available, in gigabytes (GB), for each autoscaled compute node shape. This value is only used for VMs, and is ignored for Bare Metal instances.

    Default value: 24
    Cluster Network CIDR Block

    The IPV4 CIDR blocks for your OpenShift cluster network.
    Service Network CIDR Block

    The IPV4 CIDR blocks for your OpenShift service network.
    Note

    Autoscaling is supported on OpenShift Container Platform 4.22 and later. The RHCOS worker image must match the OpenShift Container Platform minor version of the cluster. When autoscaling is enabled without static workers, control-plane nodes can receive the worker role. After autoscaled workers reach the Ready state, follow Red Hat solution 4564851 to remove the worker role from control-plane nodes.

    Compute Node Configuration

    Note: This section is visible only when the Create OpenShift Image and Instances checkbox is selected. It is not available if you select Enable OCI OpenShift Autoscaler for an Autoscaling Terraform deployment.

    Field Description
    Compute Shape

    The instance shape of the compute nodes. For more information, see Compute Shapes.

    Default value: VM.Standard.E5.Flex
    (Optional) Compute Capacity Reservation OCID

    The OCID of the capacity reservation used for compute nodes.
    Compute Node Count

    The number of compute nodes in the cluster.

    Default value: 3
    Compute Node OCPU

    The number of OCPUs available each compute node shape.

    Default value: 6
    Compute Node Memory

    The amount of memory available, in gigabytes (GB), for each compute node shape.

    Default value: 32 GB
    Compute Node Boot Volume

    The boot volume size (GB) of each compute node.

    Default value: 300 GB
    Compute Node VPU

    The number of Volume Performance Units (VPUs) applied to the volume, per 1 GB, of each compute node.

    Default value: 30
    Distribute Compute Instances Across ADs

    Enables automatic distribution of compute instances across Availability domains (ADs) in a round-robin sequence, starting from the selected AD. If unselected, all nodes are created in the selected starting AD.

    Default value: True
    (Optional) Starting AD

    Availability domain (AD) used for initial node placement. Additional nodes are automatically distributed across ADs in a round-robin sequence, starting from the selected AD.
    Distribute Compute Instances Across FDs

    Distributes compute instances across Fault Domains in a round-robin sequence. If not selected, the OCI Compute service distributes them based on shape availability and other criteria.

    Default value: True
    (Optional) Starting FD

    The fault domain used for initial compute-node placement. Additional nodes are distributed across fault domains in round-robin order, beginning with the selected fault domain.
  26. In the Agent-based Installation Advanced Configuration section, note the value in the Rendezvous IP field. This IP address is assigned to one of the control plane instances, which then acts as the bootstrap node for the cluster. You can access the bootstrap cluster node from the webserver to monitor the installation process. 
    ssh -i id_rsa core@<rendezvous_ip>
    Once connected, run the following command to monitor the logs on the bootstrap node:
    journalctl -f

    Before running the SSH command, upload your private SSH key id_rsa to the webserver.

    scp ~/.ssh/id_rsa opc@<webserver_public_ip>:/home/opc
    Note

    • The value in Rendezvous IP field must match the rendezvousIP in theagent-config.yaml.
    • Don't change any configurations in the Agent-based Installation Advanced Configuration section during this step.
  27. Select Next to review and save the changes.
  28. On the stack details page, select Apply to run another apply job for the stack. The job creates the custom software image used by the Compute instances in the cluster, and it provisions the Compute instances. After the compute instances are provisioned by the stack, the cluster installation begins automatically.

    What's Next? Follow the instructions in Verifying that your Agent-based cluster installation runs on OCI (Red Hat documentation) to verify your cluster is running. This step is performed in the OpenShift Container Platform CLI.