Select a documentation version: <button class="button button-with-icon cloud-select-dropdown__button button-primary" aria-haspopup="true">15.5.5</button>

- [15.5.5](https://docs.cloud.google.com/alloydb/omni/containers/15.5.5/docs/model-endpoint-predictions.md.txt)
- [15.5.4](https://docs.cloud.google.com/alloydb/omni/containers/15.5.4/docs/model-endpoint-predictions.md.txt)
- [15.5.2](https://docs.cloud.google.com/alloydb/omni/containers/15.5.2/docs/model-endpoint-predictions.md.txt)

<br />

> [!WARNING]
>
> **Preview
> --- Model endpoint management**
>
>
> This feature is
>
> subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the
> [Service Specific
> Terms](https://docs.cloud.google.com/terms/service-terms#1).
>
> Pre-GA features are available "as is" and might have limited support.
>
> For more information, see the
> [launch stage descriptions](https://cloud.google.com/products/#product-launch-stages).

This page describes a preview that lets you experiment with registering an AI model endpoint
and invoking predictions with Model endpoint management. For using AI models in
production environments, see [Build generative AI applications using
AlloyDB AI](https://docs.cloud.google.com/alloydb/docs/ai).

After the model endpoints are added and registered in the Model endpoint management, you can
reference them using the model ID to invoke predictions.

## Before you begin

Make sure that you have registered your model endpoint with Model endpoint management. For more information, see [Register a model endpoint with model endpoint management](https://docs.cloud.google.com/alloydb/omni/containers/15.5.5/docs/model-endpoint-register-model.md.txt)

## Invoke predictions for generic models

Use the `google_ml.predict_row()` SQL function to call a registered generic model endpoint to invoke
predictions. You can use `google_ml.predict_row()` function with any model type.

```sql
SELECT
  google_ml.predict_row(
    model_id => 'MODEL_ID',
    request_body => 'REQUEST_BODY');
```

Replace the following:

- `MODEL_ID`: the model ID you defined when registering the model endpoint.
- `REQUEST_BODY`: the parameters to the prediction function, in JSON format.

## Examples

Some examples for invoking predictions using registered model endpoints are listed in this section.

To generate predictions for a registered `gemini-pro` model endpoint, run the following statement:

```sql
    SELECT
        json_array_elements(
        google_ml.predict_row(
            model_id => 'gemini-pro',
            request_body => '{
        "contents": [
            {
                "role": "user",
                "parts": [
                    {
                        "text": "For TPCH database schema as mentioned here https://www.tpc.org/TPC_Documents_Current_Versions/pdf/TPC-H_v3.0.1.pdf , generate a SQL query to find all supplier names which are located in the India nation."
                    }
                ]
            }
        ]
        }'))-> 'candidates' -> 0 -> 'content' -> 'parts' -> 0 -> 'text';
```

To generate predictions for a registered `facebook/bart-large-mnli` model endpoint on Hugging Face, run the following statement:

```sql
  SELECT
  google_ml.predict_row(
    model_id => 'facebook/bart-large-mnli',
    request_body =>
      '{
       "inputs": "Hi, I recently bought a device from your company but it is not working as advertised and I would like to get reimbursed!",
    "parameters": {"candidate_labels": ["refund", "legal", "faq"]}
    }'
  );
```