Vertex AI V1 API - Class Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec (v1.27.0)

Reference documentation and code samples for the Vertex AI V1 API class Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec.

Configuration for Speculative Decoding.

Inherits

• Object

Extended By

• Google::Protobuf::MessageExts::ClassMethods

Includes

• Google::Protobuf::MessageExts

Methods

#draft_model_speculation

def draft_model_speculation() -> ::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::DraftModelSpeculation

Returns

• (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::DraftModelSpeculation) — draft model speculation.

  Note: The following fields are mutually exclusive: draft_model_speculation, ngram_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.

#draft_model_speculation=

def draft_model_speculation=(value) -> ::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::DraftModelSpeculation

Parameter

• value (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::DraftModelSpeculation) — draft model speculation.

  Note: The following fields are mutually exclusive: draft_model_speculation, ngram_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns

• (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::DraftModelSpeculation) — draft model speculation.

  Note: The following fields are mutually exclusive: draft_model_speculation, ngram_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.

#ngram_speculation

def ngram_speculation() -> ::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation

Returns

• (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation) — N-Gram speculation.

  Note: The following fields are mutually exclusive: ngram_speculation, draft_model_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.

#ngram_speculation=

def ngram_speculation=(value) -> ::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation

Parameter

• value (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation) — N-Gram speculation.

  Note: The following fields are mutually exclusive: ngram_speculation, draft_model_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns

• (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation) — N-Gram speculation.

  Note: The following fields are mutually exclusive: ngram_speculation, draft_model_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.

#speculative_token_count

def speculative_token_count() -> ::Integer

Returns

• (::Integer) — The number of speculative tokens to generate at each step.

#speculative_token_count=

def speculative_token_count=(value) -> ::Integer

Parameter

• value (::Integer) — The number of speculative tokens to generate at each step.

Returns

• (::Integer) — The number of speculative tokens to generate at each step.