Oracle Cloud Infrastructure Documentation

Bring Your Own Models

If you have models you want to use instead of the cached models provided by Data Science, you can bring them into AI Quick Actions from Object Storage or from Hugging Face by registering the model.

Hugging Face is an open source model repository. You can bring in models from here to use in AI Quick Actions. Hugging Face offers certain gated models that require the acceptance of user agreement. To bring a gated model from Hugging Face into AI Quick Actions, sign in to Hugging Face using the Hugging Face CLI and your Hugging Face token from a terminal inside the Notebook. This is to verify your access to the model. See the Hugging Face guides to see how to sign in with the Hugging Face CLI. If you don't have a Hugging Face token, see this Hugging Face article on security tokens to generate one.

There are two ways to register a model:
  • Register a verified model.
  • Register an unverified model.
A verified model is one the Data Science service has tested the configurations for deployment and fine tuning. An unverified model is one the service hasn't tested.
Note

The difference between a service cached model and a verified model is that, for a verified model, you must register the model in AI Quick Actions before using it.

Service Managed Inference Containers

Three inference containers are available to use with Bring Your Own Model.

For cached and verified models, Data Science has tested which inference container works best with each model and so the inference container can't be chosen. For unverified models, you must decide which inference container is most suitable for each model. Three service managed inference containers are available, one for models compatible with inference engine vLLM 0.5.3 post1 another container for models compatible with TGI 2.0.1, and the third for models compatible with inference framework llama.cpp for models in the GGUF format.

Register Verified Models

Data Science has models you can select to use that have been tested.

  • Follow the steps in Prerequisites.
    Steps 3 and 4 show different ways of registering a model. Model registration is a necessary process for a model to be brought into AI Quick Actions.
    1. Click Models if it's not already shown.
    2. Opt to import a new model from Object Storage or Hugging Face to register, or to use a ready-to-register model:
      • To import a new model, click My models.
      • To use a ready-to-register model, click Ready-to-register models,
    3. If in step 2 you clicked My models:
      1. Click Import new model.
      2. Under Model artifact, select one of:
        • Download from Hugging Face to download a model from Hugging Face.
        • I have artifacts in Object storage to download a model from Object Storage.
      3. Click Register verified model to select a model that's been tested by Oracle Data Science for deployment and fine-tuning.
      4. From the Select model list, select the model name.
      5. From the Select compartment list, select the compartment of the Object Storage bucket.
      6. From the Object storage location list, select a bucket.
        • If downloading from Object Storage, it's the bucket where the model artifact is stored.
        • If downloading from Hugging Face, it's the bucket where the model artifact is downloaded.
      7. Provide a directory path for the object storage.
      8. Click Register to register the model. When the model registration finishes, the Model Information screen is displayed. The model is included in the list of models under My models.
      9. Click Fine-tune to fine tune the model.
      10. Click Deploy to deploy the model.
    4. If in step 2 you clicked Ready-to-register models:
      1. Under Model artifact, select one of:
        • Download from Hugging Face to download a model from Hugging Face.
        • I have artifacts in Object storage to download a model from Object Storage.
      2. Click Register verified model to select a model that's been tested by Oracle Data Science for deployment and fine-tuning.
      3. From the Select model list, select the model name.
      4. From the Select compartment list, select the compartment of the Object Storage bucket..
      5. From the Object storage location list, select a bucket.
        • If downloading from Object Storage, it's the bucket where the model artifact is stored.
        • If downloading from Hugging Face, it's the bucket where the model artifact is downloaded.
      6. Provide a directory path for the object storage.
      7. Click Register to register the model. When the model registration finishes, the Model Information screen is displayed. The model is included in the list of models under My models.
      8. Click Fine-tune to fine tune the model.
      9. Click Deploy to deploy the model.

  • For a complete list of parameters and values for AI Quick Actions CLI commands, see AI Quick Actions CLI

  • This task can't be performed using the API.

Register Unverified Models

Follow these steps to use models that haven't been tested by Data Science.

  • Follow the steps in Prerequisites.
    Model registration is a necessary process for a model to be brought into AI Quick Actions.
    1. Click Models if it's not already shown.
    2. Click My models.
    3. Click Import new model.
    4. Under Model artifact, select one of:
      • Download from Hugging Face to download a model from Hugging Face.
      • I have artifacts in Object storage to download a model from Object Storage.
    5. Click Register unverified model to register an unverified model.
    6. In Model name, give the model a name with which to register it.
      Note

      For a Hugging Face model, you must enter the name exactly as it appears in Hugging Face and click Search. For example, to download the Meta-Llama-3.1-405B model, the name must be entered as meta-llama/Meta-Llama-3.1-405B.
    7. Under Inference Container, select a container from the list to use for inferencing.
    8. From the Select compartment list, select the compartment of the Object Storage bucket.
    9. From the Object storage location list, select the bucket where:
      • If downloading from Object Storage, the model artifact is stored.
      • If downloading from Hugging Face, the model is downloaded to.
    10. Provide a directory path for the object storage.
    11. (Optional) To fine-tune the model, check Enable Fine tuning.
    12. Click Register to register the model.
      When the model registration finishes, the Model Information screen is displayed. The model is included in the list of models under My models.
    13. Click Fine-tune to fine tune the model.
    14. Click Deploy to deploy the model.

  • For a complete list of parameters and values for AI Quick Actions CLI commands, see AI Quick Actions CLI

  • This task can't be performed using the API.