ModelParameter(mapping=None, *, ignore_unknown_fields=False, **kwargs)Parameters to be passed to the LLM. If not set, default values will be used.
.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields
Attributes |
|
|---|---|
| Name | Description |
temperature |
float
The temperature used for sampling. Temperature sampling occurs after both topP and topK have been applied. Valid range: [0.0, 1.0] Low temperature = less random. High temperature = more random. This field is a member of oneof_ _temperature.
|
max_decode_steps |
int
The maximum number of tokens to generate. This field is a member of oneof_ _max_decode_steps.
|
top_p |
float
If set, only the tokens comprising the top top_p probability mass are considered. If both top_p and top_k are set, top_p will be used for further refining candidates selected with top_k. Valid range: (0.0, 1.0]. Small topP = less random. Large topP = more random. This field is a member of oneof_ _top_p.
|
top_k |
int
If set, the sampling process in each step is limited to the top_k tokens with highest probabilities. Valid range: [1, 40] or 1000+. Small topK = less random. Large topK = more random. This field is a member of oneof_ _top_k.
|