OCI Data Flow Run Configuration Parameters
The OCI Data Flow run configuration properties can be parameterized when updating a task's run configuration. By parameterizing the properties, you have the flexibility to use the same task with different default parameter values at design time and runtime.
Upon parameterizing a property, Data Integration adds a parameter of type String and sets the default parameter value to the value that's currently configured for that property.
The parameter names for the properties of the OCI Data Flow service task run configuration are as follows:
| Task run property | Parameter name |
|---|---|
| Pool | OCI_DF_POOL |
| Private endpoint | OCI_DF_PRIVATE_ENDPOINT |
| Log bucket path | OCI_DF_LOG_BUCKET |
| Artifact bucket path | OCI_DF_ARTIFACT_BUCKET |
| Application compartment | OCI_DF_APP_COMPARTMENT |
| Minimum number of workers | OCI_DF_MIN_WORKERS |
| Maximum number of workers | OCI_DF_MAX_WORKERS |
| Custom OCI Data Flow configuration | OCI_DF_CUSTOM_OCI_DF_SPARK_CONFIG |
Note the following about using Data Flow run configuration parameters when creating a task:
- The parameter name and type cannot be edited.
- You can edit the parameter to add a description. A parameter description, if added, is shown as a tip in the panel that's displayed for changing default parameter values at design time or runtime.
- If you no longer want a property to be parameterized, you can remove the parameter from that property.
A task that's configured and parameterized for the Data Flow run service can be added to a pipeline. When configuring the incoming parameters in the pipeline, pipeline task, and task schedule for the pipeline task, use the following format if editing the default parameter values for OCI_DF_LOG_BUCKET and OCI_DF_ARTIFACT_BUCKET: oci://<bucket-name>@<tenancy-name>
Bucket paths are required only when you're creating a pipeline, pipeline task, and task schedule for a pipeline task with Data Flow run configuration properties that are parameterized.