Search with OpenSearch Clusters
OCI Search with OpenSearch lets you create OpenSearch clusters for your search solutions.
After you've created your OpenSearch cluster, Search with OpenSearch handles most of the management and operations of search clusters, including operations such as security updates, upgrades, resizing, and scheduled backups.
You can perform the following OpenSearch cluster tasks:
List the OpenSearch clusters in a compartment.
Create a new OpenSearch cluster.
Get an OpenSearch cluster's details.
Upgrading Search with OpenSearch Cluster Software.
Delete an OpenSearch cluster from your tenancy.
For information about scaling up your cluster configuration, see Resizing a Search with OpenSearch Cluster. For steps on how to restore your cluster from a backup, see Search with OpenSearch Automated Cluster Backups. To upgrade your existing cluster to a newer version of Open Search, see Upgrading Search with OpenSearch Cluster Software.
IAM Policy
To use Oracle Cloud Infrastructure, you must be granted security access in a policy by an administrator. This access is required whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you get a message that you don’t have permission or are unauthorized, verify with your administrator what type of access you have and which compartment to work in.
To create or manage a cluster, you need to configure permissions to grant access for users to create and manage the required Networking resources, in addition to users permissions to create and manage Search with OpenSearch resources. The Networking permissions need to be configured for the compartment that contains the Networking resources, so if the cluster is in a different compartment from the VCN and subnet, ensure that the Networking permissions are configured for the compartment containing the VCN and subnet.
The following policy example includes the required permissions for a custom group
SearchOpenSearchAdmins:
Allow group SearchOpenSearchAdmins to manage vnics in compartment <NETWORK_RESOURCES_COMPARTMENT>
Allow group SearchOpenSearchAdmins to manage vcns in compartment <NETWORK_RESOURCES_COMPARTMENT>
Allow group SearchOpenSearchAdmins to manage subnets in compartment <NETWORK_RESOURCES_COMPARTMENT>
Allow group SearchOpenSearchAdmins to use network-security-groups in compartment <NETWORK_RESOURCES_COMPARTMENT>
Allow group SearchOpenSearchAdmins to manage opensearch-family in compartment <CLUSTER_RESOURCES_COMPARTMENT>The
SearchOpenSearchAdmins group in this example refers
to a custom group that you create. See Managing
Groups for more information.The permissions to Networking resources included in this example are required as specified. You can configure the permissions for Search with OpenSearch resources, specified in the last line in this example, with more granularity.
If you're new to policies, see Getting Started with Policies and Common Policies.
Configuring the Load Balancer
When you create an OpenSearch cluster, a load balancer is automatically created as well. This load balancer handles all the requests sent to the OpenSearch API endpoint. You can select the load balancer's bandwidth size when you create the OpenSearch cluster to optimize the handling of incoming message traffic. You can also edit existing OpenSearch clusters to change their load balancer's bandwidth.
Selecting the bandwidth size is optional. If you don't specify a bandwidth size for the load balancer, a default size is set based on the OpenSearch cluster size (number of master, data, kibana and search nodes).
Machine Learning Nodes
Machine learning (ML) nodes are optimized for running machine learning workloads through the ML Commons plugin and related AI components in OpenSearch. These nodes handle compute-intensive operations such as model registration, deployment, inference, text embedding, anomaly detection, and RCA analytics.
By isolating ML activity from primary data and cluster-manager nodes, ML nodes allow the cluster to run advanced AI pipelines and inference workloads without competing for I/O or memory on indexing/search tasks.
The ML nodes are automatically assigned the dedicated role ( node.roles: ["ml"] ) and they also have an attached block volume. This role designates them as compute endpoints for ML Commons jobs and text-embedding ingest pipelines. They can process models locally or connect to remote inference endpoints, such as OCI Data Science Model Deployments or external LLMs).
ML nodes provide the following benefits:
- Performance: Offloads compute-intensive ML inference and training tasks from data and cluster-manager nodes, ensuring consistently low latency for search and indexing.
- Scalability: Supports both horizontal scaling (adding more ML nodes) and vertical scaling (increasing OCPU/memory), enabling elastic resource allocation for varying ML workloads.
- Cost Efficiency: OCI FLEX shapes can be used to optimize cost-per-inference, paying only for the required compute footprint.
- Ease of Management: The ML Commons controller automatically routes inference and training tasks to eligible ML nodes based on available memory and queue depth.
- Integration Flexibility: Seamlessly connects with text embedding ingest pipelines, vector search, anomaly detection, and agentic AI features introduced in Open Search 3.x.
ML nodes provide a scalable, isolated compute layer for AI/ML workloads, improving cluster throughput, stability, and resource efficiency.
Configuring ML Node for Your Cluster
You can configure ML nodes during cluster creation or update an existing cluster with them. You can perform horizontal and vertical resizing operations on ML nodes similar to a regular node, including both scale up and scale down.
However, an ML node scale down to 0 isn't allowed when ML activities are detected on the cluster. A request to downscale ML node to 0 fails unless these ML related activities are stopped such as undeploying an ML model. Qualified ML activities include:
- Deployed ML models
- Registered AI agents
- Registered & deployed remote models
- AI assistant root agent
- MCP tools
In absence of ML node, ML activities run on regular data nodes which automatically assume the role of ml node as is the case in clusters without dedicated ML nodes.
Cluster Settings
When creating a cluster with dedicated ML node or adding dedicated ML nodes to an existing cluster, the "only_run_on_ml_node": "true" parameter is by default set to True. This parameter automatically gets reset to False after a successful scale down of ML nodes to 0 to allow new ML workloads to run seamlessly on data node.
You can change these cluster setting parameters. However, you should clearly understand how these settings affect the cluster. If this parameter is set to True on a cluster with no dedicated ML nodes, then all ML requests fail to process. Similarly, setting this flag to False on a cluster with dedicated ML node means both ML and Data nodes processes the ML workload and that defeats the intended purpose of isolating ML workloads on dedicated ML nodes from ingestion and query workloads on data nodes.
Configurable Parameters
You can use the following parameters to configure ML nodes on your cluster:
- Node shape (for SHAPE or FLEX families): The node shape for the cluster's ML nodes.
- Node count: The number of ML nodes to add to the cluster.
- Node memory: The amount of memory for the ML nodes in gigabytes.
- Node OCPU count: The number of OCPUs to configure for the cluster's ML nodes.
- Node storage: The amount of storage in gigabytes to configure per node for the cluster's ML nodes.
GPU nodes aren't supported for either ML or data nodes.
Generating a Password Hash (CLI and Terraform Only)
If you're including the security-master-user-password-hash parameter to use with the security-master-user-name parameter, you need to generate the password hash:
- Download the following file:
- Open a command prompt and run the following command:
java -jar oci-crypto-common.jar pbkdf2_stretch_1000 <password-in-plain-text>where
<password-in-plain-text>is a password of your choosing.You must have Java installed on your computer before running this command.
- Record the generated password hash.
- Include the generated password hash as the value for the
security-master-user-password-hashparameter.Use single quotes around the password. For example:
oci opensearch cluster create ... --security-master-user-name MyUser --security-master-user-password-hash 'pbkdf2_...MNUT5No='
Tagging Resources
You can apply tags to your resources to help you organize them according to your business needs. You can apply tags at the time you create a resource, or you can update the resource later with the wanted tags. For general information about applying tags, see Overview of Tagging.