# LlmModelSettings

- [JSON representation](https://docs.cloud.google.com/dialogflow/cx/docs/reference/rest/v3beta1/LlmModelSettings#SCHEMA_REPRESENTATION)
- [Parameters](https://docs.cloud.google.com/dialogflow/cx/docs/reference/rest/v3beta1/LlmModelSettings#Parameters)
  - [JSON representation](https://docs.cloud.google.com/dialogflow/cx/docs/reference/rest/v3beta1/LlmModelSettings#Parameters.SCHEMA_REPRESENTATION)
- [InputTokenLimit](https://docs.cloud.google.com/dialogflow/cx/docs/reference/rest/v3beta1/LlmModelSettings#InputTokenLimit)
- [OutputTokenLimit](https://docs.cloud.google.com/dialogflow/cx/docs/reference/rest/v3beta1/LlmModelSettings#OutputTokenLimit)

Settings for LLM models.

| JSON representation |
|---|
| ``` { "model": string, "promptText": string, "parameters": { object (`https://docs.cloud.google.com/dialogflow/cx/docs/reference/rest/v3beta1/LlmModelSettings#Parameters`) } } ``` |

| Fields ||
|---|---|
| `model` | `string` The selected LLM model. |
| `promptText` | `string` The custom prompt to use. |
| `parameters` | ``object (`https://docs.cloud.google.com/dialogflow/cx/docs/reference/rest/v3beta1/LlmModelSettings#Parameters`)`` Generative model parameters. |

## Parameters

Generative model parameters to control the model behavior.

| JSON representation |
|---|
| ``` { "temperature": number, "inputTokenLimit": enum (`https://docs.cloud.google.com/dialogflow/cx/docs/reference/rest/v3beta1/LlmModelSettings#InputTokenLimit`), "outputTokenLimit": enum (`https://docs.cloud.google.com/dialogflow/cx/docs/reference/rest/v3beta1/LlmModelSettings#OutputTokenLimit`) } ``` |

| Fields ||
|---|---|
| `temperature` | `number` The temperature used for sampling during response generation. Value ranges from 0 to 1. Temperature controls the degree of randomness in token selection. Lower temperature means less randomness, while higher temperature means more randomness. Valid range: \[0.0, 1.0\] |
| `inputTokenLimit` | ``enum (`https://docs.cloud.google.com/dialogflow/cx/docs/reference/rest/v3beta1/LlmModelSettings#InputTokenLimit`)`` The input token limit. This setting is currently only supported by playbooks. |
| `outputTokenLimit` | ``enum (`https://docs.cloud.google.com/dialogflow/cx/docs/reference/rest/v3beta1/LlmModelSettings#OutputTokenLimit`)`` The output token limit. This setting is currently only supported by playbooks. Only one of outputTokenLimit and max_output_tokens is allowed to be set. |

## InputTokenLimit

The input token limits for 1 LLM call. For the limit of each model, see <https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models> for more information.

| Enums ||
|---|---|
| `INPUT_TOKEN_LIMIT_UNSPECIFIED` | Limit not specified. Treated as 'INPUT_TOKEN_LIMIT_SHORT'. |
| `INPUT_TOKEN_LIMIT_SHORT` | Input token limit up to 8k. |
| `INPUT_TOKEN_LIMIT_MEDIUM` | Input token limit up to 32k. |
| `INPUT_TOKEN_LIMIT_LONG` | Input token limit up to 100k. |

## OutputTokenLimit

The output token limits for 1 LLM call. The limits are subject to change. For the limit of each model, see <https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models> for more information.

| Enums ||
|---|---|
| `OUTPUT_TOKEN_LIMIT_UNSPECIFIED` | Limit not specified. |
| `OUTPUT_TOKEN_LIMIT_SHORT` | Input token limit up to 512 tokens. |
| `OUTPUT_TOKEN_LIMIT_MEDIUM` | Input token limit up to 1k. |
| `OUTPUT_TOKEN_LIMIT_LONG` | Input token limit up to 2k. |