Class: OCI::GenerativeAiInference::Models::CohereLlmInferenceRequest
- Inherits:
-
LlmInferenceRequest
- Object
- LlmInferenceRequest
- OCI::GenerativeAiInference::Models::CohereLlmInferenceRequest
- Defined in:
- lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb
Overview
Details for the text generation request for Cohere models.
Constant Summary collapse
- RETURN_LIKELIHOODS_ENUM =
[ RETURN_LIKELIHOODS_NONE = 'NONE'.freeze, RETURN_LIKELIHOODS_ALL = 'ALL'.freeze, RETURN_LIKELIHOODS_GENERATION = 'GENERATION'.freeze ].freeze
- TRUNCATE_ENUM =
[ TRUNCATE_NONE = 'NONE'.freeze, TRUNCATE_START = 'START'.freeze, TRUNCATE_END = 'END'.freeze ].freeze
Constants inherited from LlmInferenceRequest
LlmInferenceRequest::RUNTIME_TYPE_ENUM
Instance Attribute Summary collapse
-
#frequency_penalty ⇒ Float
To reduce repetitiveness of generated tokens, this number penalizes new tokens based on their frequency in the generated text so far.
-
#is_echo ⇒ BOOLEAN
Whether or not to return the user prompt in the response.
-
#is_stream ⇒ BOOLEAN
Whether to stream back partial progress.
-
#max_tokens ⇒ Integer
The maximum number of tokens to predict for each response.
-
#num_generations ⇒ Integer
The number of generated texts that will be returned.
-
#presence_penalty ⇒ Float
To reduce repetitiveness of generated tokens, this number penalizes new tokens based on whether they've appeared in the generated text so far.
-
#prompt ⇒ String
[Required] Represents the prompt to be completed.
-
#return_likelihoods ⇒ String
Specifies how and if the token likelihoods are returned with the response.
-
#stop_sequences ⇒ Array<String>
The generated text is cut at the end of the earliest occurrence of this stop sequence.
-
#temperature ⇒ Float
A number that sets the randomness of the generated output.
-
#top_k ⇒ Integer
An integer that sets up the model to use only the top k most likely tokens in the generated output.
-
#top_p ⇒ Float
If set to a probability 0.0 < p < 1.0, it ensures that only the most likely tokens, with total probability mass of p, are considered for generation at each step.
-
#truncate ⇒ String
For an input that's longer than the maximum token length, specifies which part of the input text will be truncated.
Attributes inherited from LlmInferenceRequest
Class Method Summary collapse
-
.attribute_map ⇒ Object
Attribute mapping from ruby-style variable name to JSON key.
-
.swagger_types ⇒ Object
Attribute type mapping.
Instance Method Summary collapse
-
#==(other) ⇒ Object
Checks equality by comparing each attribute.
-
#build_from_hash(attributes) ⇒ Object
Builds the object from hash.
- #eql?(other) ⇒ Boolean
-
#hash ⇒ Fixnum
Calculates hash code according to all attributes.
-
#initialize(attributes = {}) ⇒ CohereLlmInferenceRequest
constructor
Initializes the object.
-
#to_hash ⇒ Hash
Returns the object in the form of hash.
-
#to_s ⇒ String
Returns the string representation of the object.
Methods inherited from LlmInferenceRequest
Constructor Details
#initialize(attributes = {}) ⇒ CohereLlmInferenceRequest
Initializes the object
152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 152 def initialize(attributes = {}) return unless attributes.is_a?(Hash) attributes['runtimeType'] = 'COHERE' super(attributes) # convert string to symbol for hash key attributes = attributes.each_with_object({}) { |(k, v), h| h[k.to_sym] = v } self.prompt = attributes[:'prompt'] if attributes[:'prompt'] self.is_stream = attributes[:'isStream'] unless attributes[:'isStream'].nil? self.is_stream = false if is_stream.nil? && !attributes.key?(:'isStream') # rubocop:disable Style/StringLiterals raise 'You cannot provide both :isStream and :is_stream' if attributes.key?(:'isStream') && attributes.key?(:'is_stream') self.is_stream = attributes[:'is_stream'] unless attributes[:'is_stream'].nil? self.is_stream = false if is_stream.nil? && !attributes.key?(:'isStream') && !attributes.key?(:'is_stream') # rubocop:disable Style/StringLiterals self.num_generations = attributes[:'numGenerations'] if attributes[:'numGenerations'] raise 'You cannot provide both :numGenerations and :num_generations' if attributes.key?(:'numGenerations') && attributes.key?(:'num_generations') self.num_generations = attributes[:'num_generations'] if attributes[:'num_generations'] self.is_echo = attributes[:'isEcho'] unless attributes[:'isEcho'].nil? self.is_echo = false if is_echo.nil? && !attributes.key?(:'isEcho') # rubocop:disable Style/StringLiterals raise 'You cannot provide both :isEcho and :is_echo' if attributes.key?(:'isEcho') && attributes.key?(:'is_echo') self.is_echo = attributes[:'is_echo'] unless attributes[:'is_echo'].nil? self.is_echo = false if is_echo.nil? && !attributes.key?(:'isEcho') && !attributes.key?(:'is_echo') # rubocop:disable Style/StringLiterals self.max_tokens = attributes[:'maxTokens'] if attributes[:'maxTokens'] raise 'You cannot provide both :maxTokens and :max_tokens' if attributes.key?(:'maxTokens') && attributes.key?(:'max_tokens') self.max_tokens = attributes[:'max_tokens'] if attributes[:'max_tokens'] self.temperature = attributes[:'temperature'] if attributes[:'temperature'] self.temperature = 1.0 if temperature.nil? && !attributes.key?(:'temperature') # rubocop:disable Style/StringLiterals self.top_k = attributes[:'topK'] if attributes[:'topK'] raise 'You cannot provide both :topK and :top_k' if attributes.key?(:'topK') && attributes.key?(:'top_k') self.top_k = attributes[:'top_k'] if attributes[:'top_k'] self.top_p = attributes[:'topP'] if attributes[:'topP'] self.top_p = 0.75 if top_p.nil? && !attributes.key?(:'topP') # rubocop:disable Style/StringLiterals raise 'You cannot provide both :topP and :top_p' if attributes.key?(:'topP') && attributes.key?(:'top_p') self.top_p = attributes[:'top_p'] if attributes[:'top_p'] self.top_p = 0.75 if top_p.nil? && !attributes.key?(:'topP') && !attributes.key?(:'top_p') # rubocop:disable Style/StringLiterals self.frequency_penalty = attributes[:'frequencyPenalty'] if attributes[:'frequencyPenalty'] self.frequency_penalty = 0.0 if frequency_penalty.nil? && !attributes.key?(:'frequencyPenalty') # rubocop:disable Style/StringLiterals raise 'You cannot provide both :frequencyPenalty and :frequency_penalty' if attributes.key?(:'frequencyPenalty') && attributes.key?(:'frequency_penalty') self.frequency_penalty = attributes[:'frequency_penalty'] if attributes[:'frequency_penalty'] self.frequency_penalty = 0.0 if frequency_penalty.nil? && !attributes.key?(:'frequencyPenalty') && !attributes.key?(:'frequency_penalty') # rubocop:disable Style/StringLiterals self.presence_penalty = attributes[:'presencePenalty'] if attributes[:'presencePenalty'] self.presence_penalty = 0.0 if presence_penalty.nil? && !attributes.key?(:'presencePenalty') # rubocop:disable Style/StringLiterals raise 'You cannot provide both :presencePenalty and :presence_penalty' if attributes.key?(:'presencePenalty') && attributes.key?(:'presence_penalty') self.presence_penalty = attributes[:'presence_penalty'] if attributes[:'presence_penalty'] self.presence_penalty = 0.0 if presence_penalty.nil? && !attributes.key?(:'presencePenalty') && !attributes.key?(:'presence_penalty') # rubocop:disable Style/StringLiterals self.stop_sequences = attributes[:'stopSequences'] if attributes[:'stopSequences'] raise 'You cannot provide both :stopSequences and :stop_sequences' if attributes.key?(:'stopSequences') && attributes.key?(:'stop_sequences') self.stop_sequences = attributes[:'stop_sequences'] if attributes[:'stop_sequences'] self.return_likelihoods = attributes[:'returnLikelihoods'] if attributes[:'returnLikelihoods'] self.return_likelihoods = "NONE" if return_likelihoods.nil? && !attributes.key?(:'returnLikelihoods') # rubocop:disable Style/StringLiterals raise 'You cannot provide both :returnLikelihoods and :return_likelihoods' if attributes.key?(:'returnLikelihoods') && attributes.key?(:'return_likelihoods') self.return_likelihoods = attributes[:'return_likelihoods'] if attributes[:'return_likelihoods'] self.return_likelihoods = "NONE" if return_likelihoods.nil? && !attributes.key?(:'returnLikelihoods') && !attributes.key?(:'return_likelihoods') # rubocop:disable Style/StringLiterals self.truncate = attributes[:'truncate'] if attributes[:'truncate'] self.truncate = "NONE" if truncate.nil? && !attributes.key?(:'truncate') # rubocop:disable Style/StringLiterals end |
Instance Attribute Details
#frequency_penalty ⇒ Float
To reduce repetitiveness of generated tokens, this number penalizes new tokens based on their frequency in the generated text so far. Greater numbers encourage the model to use new tokens, while lower numbers encourage the model to repeat the tokens. Set to 0 to disable.
68 69 70 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 68 def frequency_penalty @frequency_penalty end |
#is_echo ⇒ BOOLEAN
Whether or not to return the user prompt in the response. This option only applies to non-stream results.
38 39 40 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 38 def is_echo @is_echo end |
#is_stream ⇒ BOOLEAN
Whether to stream back partial progress. If set, tokens are sent as data-only server-sent events as they become available.
30 31 32 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 30 def is_stream @is_stream end |
#max_tokens ⇒ Integer
The maximum number of tokens to predict for each response. Includes input plus output tokens.
42 43 44 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 42 def max_tokens @max_tokens end |
#num_generations ⇒ Integer
The number of generated texts that will be returned.
34 35 36 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 34 def num_generations @num_generations end |
#presence_penalty ⇒ Float
To reduce repetitiveness of generated tokens, this number penalizes new tokens based on whether they've appeared in the generated text so far. Greater numbers encourage the model to use new tokens, while lower numbers encourage the model to repeat the tokens.
Similar to frequency penalty, a penalty is applied to previously present tokens, except that this penalty is applied equally to all tokens that have already appeared, regardless of how many times they've appeared. Set to 0 to disable.
75 76 77 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 75 def presence_penalty @presence_penalty end |
#prompt ⇒ String
[Required] Represents the prompt to be completed. The trailing white spaces are trimmed before completion.
26 27 28 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 26 def prompt @prompt end |
#return_likelihoods ⇒ String
Specifies how and if the token likelihoods are returned with the response.
83 84 85 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 83 def return_likelihoods @return_likelihoods end |
#stop_sequences ⇒ Array<String>
The generated text is cut at the end of the earliest occurrence of this stop sequence. The generated text will include this stop sequence.
79 80 81 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 79 def stop_sequences @stop_sequences end |
#temperature ⇒ Float
A number that sets the randomness of the generated output. A lower temperature means a less random generations.
Use lower numbers for tasks with a correct answer such as question answering or summarizing. High temperatures can generate hallucinations or factually incorrect information. Start with temperatures lower than 1.0 and increase the temperature for more creative outputs, as you regenerate the prompts to refine the outputs.
49 50 51 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 49 def temperature @temperature end |
#top_k ⇒ Integer
An integer that sets up the model to use only the top k most likely tokens in the generated output. A higher k introduces more randomness into the output making the output text sound more natural. Default value is 0 which disables this method and considers all tokens. To set a number for the likely tokens, choose an integer between 1 and 500.
If also using top p, then the model considers only the top tokens whose probabilities add up to p percent and ignores the rest of the k tokens. For example, if k is 20, but the probabilities of the top 10 add up to .75, then only the top 10 tokens are chosen.
56 57 58 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 56 def top_k @top_k end |
#top_p ⇒ Float
If set to a probability 0.0 < p < 1.0, it ensures that only the most likely tokens, with total probability mass of p, are considered for generation at each step.
To eliminate tokens with low likelihood, assign p a minimum percentage for the next token's likelihood. For example, when p is set to 0.75, the model eliminates the bottom 25 percent for the next token. Set to 1.0 to consider all tokens and set to 0 to disable. If both k and p are enabled, p acts after k.
63 64 65 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 63 def top_p @top_p end |
#truncate ⇒ String
For an input that's longer than the maximum token length, specifies which part of the input text will be truncated.
87 88 89 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 87 def truncate @truncate end |
Class Method Details
.attribute_map ⇒ Object
Attribute mapping from ruby-style variable name to JSON key.
90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 90 def self.attribute_map { # rubocop:disable Style/SymbolLiteral 'runtime_type': :'runtimeType', 'prompt': :'prompt', 'is_stream': :'isStream', 'num_generations': :'numGenerations', 'is_echo': :'isEcho', 'max_tokens': :'maxTokens', 'temperature': :'temperature', 'top_k': :'topK', 'top_p': :'topP', 'frequency_penalty': :'frequencyPenalty', 'presence_penalty': :'presencePenalty', 'stop_sequences': :'stopSequences', 'return_likelihoods': :'returnLikelihoods', 'truncate': :'truncate' # rubocop:enable Style/SymbolLiteral } end |
.swagger_types ⇒ Object
Attribute type mapping.
112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 112 def self.swagger_types { # rubocop:disable Style/SymbolLiteral 'runtime_type': :'String', 'prompt': :'String', 'is_stream': :'BOOLEAN', 'num_generations': :'Integer', 'is_echo': :'BOOLEAN', 'max_tokens': :'Integer', 'temperature': :'Float', 'top_k': :'Integer', 'top_p': :'Float', 'frequency_penalty': :'Float', 'presence_penalty': :'Float', 'stop_sequences': :'Array<String>', 'return_likelihoods': :'String', 'truncate': :'String' # rubocop:enable Style/SymbolLiteral } end |
Instance Method Details
#==(other) ⇒ Object
Checks equality by comparing each attribute.
266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 266 def ==(other) return true if equal?(other) self.class == other.class && runtime_type == other.runtime_type && prompt == other.prompt && is_stream == other.is_stream && num_generations == other.num_generations && is_echo == other.is_echo && max_tokens == other.max_tokens && temperature == other.temperature && top_k == other.top_k && top_p == other.top_p && frequency_penalty == other.frequency_penalty && presence_penalty == other.presence_penalty && stop_sequences == other.stop_sequences && return_likelihoods == other.return_likelihoods && truncate == other.truncate end |
#build_from_hash(attributes) ⇒ Object
Builds the object from hash
309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 309 def build_from_hash(attributes) return nil unless attributes.is_a?(Hash) self.class.swagger_types.each_pair do |key, type| if type =~ /^Array<(.*)>/i # check to ensure the input is an array given that the the attribute # is documented as an array but the input is not if attributes[self.class.attribute_map[key]].is_a?(Array) public_method("#{key}=").call( attributes[self.class.attribute_map[key]] .map { |v| OCI::Internal::Util.convert_to_type(Regexp.last_match(1), v) } ) end elsif !attributes[self.class.attribute_map[key]].nil? public_method("#{key}=").call( OCI::Internal::Util.convert_to_type(type, attributes[self.class.attribute_map[key]]) ) end # or else data not found in attributes(hash), not an issue as the data can be optional end self end |
#eql?(other) ⇒ Boolean
289 290 291 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 289 def eql?(other) self == other end |
#hash ⇒ Fixnum
Calculates hash code according to all attributes.
298 299 300 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 298 def hash [runtime_type, prompt, is_stream, num_generations, is_echo, max_tokens, temperature, top_k, top_p, frequency_penalty, presence_penalty, stop_sequences, return_likelihoods, truncate].hash end |
#to_hash ⇒ Hash
Returns the object in the form of hash
342 343 344 345 346 347 348 349 350 351 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 342 def to_hash hash = {} self.class.attribute_map.each_pair do |attr, param| value = public_method(attr).call next if value.nil? && !instance_variable_defined?("@#{attr}") hash[param] = _to_hash(value) end hash end |
#to_s ⇒ String
Returns the string representation of the object
336 337 338 |
# File 'lib/oci/generative_ai_inference/models/cohere_llm_inference_request.rb', line 336 def to_s to_hash.to_s end |