Select a documentation version: <button class="button button-with-icon cloud-select-dropdown__button button-primary" aria-haspopup="true">18.4.0</button>

- [Current (18.4.0)](https://docs.cloud.google.com/alloydb/omni/containers/current/docs/ai/model-endpoint-predictions.md.txt)
- [18.4.0](https://docs.cloud.google.com/alloydb/omni/containers/18.4.0/docs/ai/model-endpoint-predictions.md.txt)
- [18.3.0](https://docs.cloud.google.com/alloydb/omni/containers/18.3.0/docs/ai/model-endpoint-predictions.md.txt)
- [18.1.0](https://docs.cloud.google.com/alloydb/omni/containers/18.1.0/docs/ai/model-endpoint-predictions.md.txt)
- [17.10.0](https://docs.cloud.google.com/alloydb/omni/containers/17.10.0/docs/ai/model-endpoint-predictions.md.txt)
- [17.9.0](https://docs.cloud.google.com/alloydb/omni/containers/17.9.0/docs/ai/model-endpoint-predictions.md.txt)
- [17.7.0](https://docs.cloud.google.com/alloydb/omni/containers/17.7.0/docs/ai/model-endpoint-predictions.md.txt)
- [17.5.0](https://docs.cloud.google.com/alloydb/omni/containers/17.5.0/docs/ai/model-endpoint-predictions.md.txt)
- [16.14.0](https://docs.cloud.google.com/alloydb/omni/containers/16.14.0/docs/ai/model-endpoint-predictions.md.txt)
- [16.13.0](https://docs.cloud.google.com/alloydb/omni/containers/16.13.0/docs/ai/model-endpoint-predictions.md.txt)
- [16.11.0](https://docs.cloud.google.com/alloydb/omni/containers/16.11.0/docs/ai/model-endpoint-predictions.md.txt)
- [16.9.0](https://docs.cloud.google.com/alloydb/omni/containers/16.9.0/docs/ai/model-endpoint-predictions.md.txt)
- [16.8.0](https://docs.cloud.google.com/alloydb/omni/containers/16.8.0/docs/ai/model-endpoint-predictions.md.txt)
- [16.3.0](https://docs.cloud.google.com/alloydb/omni/containers/16.3.0/docs/ai/model-endpoint-predictions.md.txt)
- [15.18.0](https://docs.cloud.google.com/alloydb/omni/containers/15.18.0/docs/ai/model-endpoint-predictions.md.txt)
- [15.17.0](https://docs.cloud.google.com/alloydb/omni/containers/15.17.0/docs/ai/model-endpoint-predictions.md.txt)
- [15.15.0](https://docs.cloud.google.com/alloydb/omni/containers/15.15.0/docs/ai/model-endpoint-predictions.md.txt)
- [15.13.0](https://docs.cloud.google.com/alloydb/omni/containers/15.13.0/docs/ai/model-endpoint-predictions.md.txt)
- [15.12.0](https://docs.cloud.google.com/alloydb/omni/containers/15.12.0/docs/ai/model-endpoint-predictions.md.txt)
- [15.7.1](https://docs.cloud.google.com/alloydb/omni/containers/15.7.1/docs/ai/model-endpoint-predictions.md.txt)
- [15.7.0](https://docs.cloud.google.com/alloydb/omni/containers/15.7.0/docs/ai/model-endpoint-predictions.md.txt)

<br />

This page describes how to invoke predictions using the model endpoints registered in the Model endpoint management.

<br />

## Before you begin

Make sure that you have registered your model endpoint with Model endpoint management.
For more information, see [Register and call remote AI models in AlloyDB Omni](https://docs.cloud.google.com/alloydb/omni/containers/current/docs/ai/register-model-endpoint.md.txt).

## Invoke predictions for generic models

Use the `google_ml.predict_row()` SQL function to call a registered generic model endpoint to invoke
predictions.

```sql
SELECT
  google_ml.predict_row(
    model_id => 'MODEL_ID',
    request_body => 'REQUEST_BODY');
```

Replace the following:

- `MODEL_ID`: the model ID you defined when registering the model endpoint.
- `REQUEST_BODY`: the parameters to the prediction function, in JSON format.

## Examples

This section includes some examples for invoking predictions using registered model endpoints.

To generate predictions for a registered `gemini-2.5-flash:streamGenerateContent` model endpoint, run the following statement:

```sql
  SELECT
  json_array_elements( google_ml.predict_row( model_id => 'gemini-2.5-flash:streamGenerateContent',
      request_body => '{ "contents": [ { "role": "user", "parts": [ { "text": "For TPCH database schema as mentioned here https://www.tpc.org/TPC_Documents_Current_Versions/pdf/TPC-H_v3.0.1.pdf , generate a SQL query to find all supplier names which are located in the India nation." } ] } ] }'))-> 'candidates' -> 0 -> 'content' -> 'parts' -> 0 -> 'text';
```

To generate predictions for a registered `facebook/bart-large-mnli` model endpoint on Hugging Face, run the following statement:

```sql
  SELECT
  google_ml.predict_row(
    model_id => 'facebook/bart-large-mnli',
    request_body =>
      '{
       "inputs": "Hi, I recently bought a device from your company but it is not working as advertised and I would like to get reimbursed!",
    "parameters": {"candidate_labels": ["refund", "legal", "faq"]}
    }'
  );
```

To generate predictions for a registered Anthropic `claude-3-haiku-20240307` model endpoint, run the following statement:

```sql
  SELECT
  google_ml.predict_row('claude-haiku', '{
    "model": "claude-3-haiku-20240307",
    "max_tokens": 1024,
    "messages": [
        {"role": "user", "content": "Hello, world"}
    ]
  }');
```