Oracle Cloud Infrastructure Documentation

Bring Your Own Models

If you have models you want to use instead of the service cached models provided by Data Science, you can bring them into AI Quick Actions from Object Storage or from Hugging Face by registering the model.

Hugging Face is an open source model repository. You can bring in models from here to use in AI Quick Actions. Hugging Face offers certain gated models that require the acceptance of user agreement. To bring a gated model from Hugging Face into AI Quick Actions, sign in to Hugging Face using the Hugging Face CLI and your Hugging Face token from a terminal inside the Notebook. This is to verify your access to the model. See the Hugging Face guides to see how to sign in with the Hugging Face CLI. If you don't have a Hugging Face token, see this Hugging Face article on security tokens to generate one. If you try to register a gated model which you haven't been granted access to in Hugging Face or fail to sign in with the Hugging Face CLI, the registration process fails.

There are two ways to register a model:
  • Register service verified model.
  • Register any model.
A service verified model is one the Data Science service has tested the configurations for deployment and fine tuning.
Note

The difference between a service cached model and a verified model is that, for a verified model, you must register the model in AI Quick Actions before using it.

Service Managed Inference Containers

Three inference containers are available to use with Bring Your Own Model.

For cached and verified models, Data Science has tested which inference container works best with each model and so the inference container can't be chosen. For unverified models, you must decide which inference container is most suitable for each model. Three service managed inference containers are available:
  • for models compatible with inference engine vLLM 0.6.2
  • for models compatible with TGI 2.0.1
  • for models compatible with inference framework llama.cpp
for models in the GGUF format.

Register Service Verified Models

Data Science has models you can select to use that have been tested.

  • Follow the steps in Prerequisites.
    Steps 3 and 4 show different ways of registering a model. Model registration is a necessary process for a model to be brought into AI Quick Actions.
    1. Click Models if it's not already shown.
    2. Opt to import a new model from Object Storage or Hugging Face to register, or to use a service verified model:
      • To import a new model, click My models.
      • To use a service verified model, click a model card with the Ready-to-register tag.
    3. If in step 2 you clicked My models:
      1. Click Import new model.
      2. Under Model artifact, select one of:
        • Download from Hugging Face to download a model from Hugging Face.
        • I have artifacts in Object storage to download a model from Object Storage.
      3. Click Register service verified model to select a model that's been tested by Oracle Data Science for deployment and fine-tuning.
      4. From the Select model list, select the model name.
      5. From the Select compartment list, select the compartment of the Object Storage bucket.
      6. From the Object storage location list, select a bucket.
        • If downloading from Object Storage, it's the bucket where the model artifact is stored.
        • If downloading from Hugging Face, it's the bucket where the model artifact is downloaded.
      7. Provide a directory path for the object storage.
      8. Click Register to register the model. When the model registration finishes, the Model Information screen is displayed. The model is included in the list of models under My models.
      9. Click Fine-tune to fine tune the model.
      10. Click Deploy to deploy the model.
    4. If in step 2 you clicked a model card with the Ready-to-register tag:
      1. Click register in the upper right corner of model information page.
      2. Under Model artifact, select one of:
        • Download from Hugging Face to download a model from Hugging Face.
        • I have artifacts in Object storage to download a model from Object Storage.
      3. From the Select compartment list, select the compartment of the Object Storage bucket..
      4. From the Object storage location list, select a bucket.
        • If downloading from Object Storage, it's the bucket where the model artifact is stored.
        • If downloading from Hugging Face, it's the bucket where the model artifact is downloaded.
      5. Provide a directory path for the object storage.
      6. Click Register to register the model. When the model registration finishes, the Model Information screen is displayed. The model is included in the list of models under My models.
      7. Click Fine-tune to fine tune the model.
      8. Click Deploy to deploy the model.

  • For a complete list of parameters and values for AI Quick Actions CLI commands, see AI Quick Actions CLI.

  • This task can't be performed using the API.

Register Any Model

Follow these steps to use models that haven't been tested by Data Science.

  • Follow the steps in Prerequisites.
    Model registration is a necessary process for a model to be brought into AI Quick Actions.
    1. Click Models if it's not already shown.
    2. Click My models.
    3. Click Import new model.
    4. Under Model artifact, select one of:
      • Download from Hugging Face to download a model from Hugging Face.
      • I have artifacts in Object storage to download a model from Object Storage.
    5. Click Register any model to register a model.
    6. In Model name, give the model a name with which to register it.
    7. Under Inference Container, select a container from the list to use for inferencing.
    8. From the Select compartment list, select the compartment of the Object Storage bucket.
    9. From the Object storage location list, select the bucket where:
      • If downloading from Object Storage, the model artifact is stored.
      • If downloading from Hugging Face, the model is downloaded to.
    10. Provide a directory path for the object storage.
    11. (Optional) To fine-tune the model, check Enable Fine tuning.
    12. Click Register to register the model.
      When the model registration finishes, the Model Information screen is displayed. The model is included in the list of models under My models.
    13. Click Fine-tune to fine tune the model.
    14. Click Deploy to deploy the model.

  • For a complete list of parameters and values for AI Quick Actions CLI commands, see AI Quick Actions CLI.

  • This task can't be performed using the API.