Select a documentation version: <button class="button button-with-icon cloud-select-dropdown__button button-primary" aria-haspopup="true">16.8.0</button>

- [Current (18.4.0)](https://docs.cloud.google.com/alloydb/omni/containers/current/docs/reference/model-endpoint.md.txt)
- [18.4.0](https://docs.cloud.google.com/alloydb/omni/containers/18.4.0/docs/reference/model-endpoint.md.txt)
- [18.3.0](https://docs.cloud.google.com/alloydb/omni/containers/18.3.0/docs/reference/model-endpoint.md.txt)
- [18.1.0](https://docs.cloud.google.com/alloydb/omni/containers/18.1.0/docs/reference/model-endpoint.md.txt)
- [17.10.0](https://docs.cloud.google.com/alloydb/omni/containers/17.10.0/docs/reference/model-endpoint.md.txt)
- [17.9.0](https://docs.cloud.google.com/alloydb/omni/containers/17.9.0/docs/reference/model-endpoint.md.txt)
- [17.7.0](https://docs.cloud.google.com/alloydb/omni/containers/17.7.0/docs/reference/model-endpoint.md.txt)
- [17.5.0](https://docs.cloud.google.com/alloydb/omni/containers/17.5.0/docs/reference/model-endpoint.md.txt)
- [16.14.0](https://docs.cloud.google.com/alloydb/omni/containers/16.14.0/docs/reference/model-endpoint.md.txt)
- [16.13.0](https://docs.cloud.google.com/alloydb/omni/containers/16.13.0/docs/reference/model-endpoint.md.txt)
- [16.11.0](https://docs.cloud.google.com/alloydb/omni/containers/16.11.0/docs/reference/model-endpoint.md.txt)
- [16.9.0](https://docs.cloud.google.com/alloydb/omni/containers/16.9.0/docs/reference/model-endpoint.md.txt)
- [16.8.0](https://docs.cloud.google.com/alloydb/omni/containers/16.8.0/docs/reference/model-endpoint.md.txt)
- [16.3.0](https://docs.cloud.google.com/alloydb/omni/containers/16.3.0/docs/reference/model-endpoint.md.txt)
- [15.18.0](https://docs.cloud.google.com/alloydb/omni/containers/15.18.0/docs/reference/model-endpoint.md.txt)
- [15.17.0](https://docs.cloud.google.com/alloydb/omni/containers/15.17.0/docs/reference/model-endpoint.md.txt)
- [15.15.0](https://docs.cloud.google.com/alloydb/omni/containers/15.15.0/docs/reference/model-endpoint.md.txt)
- [15.13.0](https://docs.cloud.google.com/alloydb/omni/containers/15.13.0/docs/reference/model-endpoint.md.txt)
- [15.12.0](https://docs.cloud.google.com/alloydb/omni/containers/15.12.0/docs/reference/model-endpoint.md.txt)
- [15.7.1](https://docs.cloud.google.com/alloydb/omni/containers/15.7.1/docs/reference/model-endpoint.md.txt)
- [15.7.0](https://docs.cloud.google.com/alloydb/omni/containers/15.7.0/docs/reference/model-endpoint.md.txt)

<br />

This page lists parameters for different functions provided by the `google_ml_integration` extension to register and manage model endpoints, and secrets with model endpoint management.

<br />

You must set the
`google_ml_integration.enable_model_support` database flag to `on` before you
can start using the extension.

For more information, see [Use Model endpoint management
with AlloyDB Omni for AI
models](https://docs.cloud.google.com/alloydb/omni/containers/16.8.0/docs/ai/model-endpoint-overview).

## Models

Use this reference to understand parameters for functions that let you manage
model endpoints.

### `google_ml.create_model()` function

The following shows how to call the `google_ml.create_model()` SQL function used
to register model endpoint metadata:

```sql
  CALL
    google_ml.create_model(
      model_id => 'MODEL_ID',
      model_request_url => 'REQUEST_URL',
      model_provider => 'PROVIDER_ID',
      model_type => 'MODEL_TYPE',
      model_qualified_name => 'MODEL_QUALIFIED_NAME',
      model_auth_type => 'AUTH_TYPE',
      model_auth_id => 'AUTH_ID',
      generate_headers_fn => 'GENERATE_HEADER_FUNCTION',
      model_in_transform_fn => 'INPUT_TRANSFORM_FUNCTION',
      model_out_transform_fn => 'OUTPUT_TRANSFORM_FUNCTION');
```

| Parameter | Required | Description |
|---|---|---|
| `MODEL_ID` | required for all model endpoints | A unique ID for the model endpoint that you define. |
| `REQUEST_URL` | optional for other text embedding model endpoints with built-in support | The model-specific endpoint when adding other text embedding and generic model endpoints. For AlloyDB for PostgreSQL, provide an `https` URL. The request URL that the function generates for built-in model endpoints refers to your cluster's project and region or location. If you want to refer to another project, then ensure that you specify the `model_request_url` explicitly. For a list of request URLs for Vertex AI model endpoints, see [Vertex AI model endpoints request URL](https://docs.cloud.google.com/alloydb/omni/containers/16.8.0/docs/reference/model-endpoint.md.txt#vertex-ai). For custom hosted model endpoints, ensure that the model endpoint is accessible from the network where AlloyDB is located. |
| `PROVIDER_ID` | required for text embedding model endpoints with built-in support | The provider of the model endpoint. The default value is `custom`. Set to one of the following: - `google` for Vertex AI model endpoints - `open_ai` for OpenAI model endpoints - `hugging_face` for Hugging Face model endpoints - `anthropic` for Anthropic model endpoints - `custom` for other providers |
| `MODEL_TYPE` | optional for generic model endpoints | The model type. Set to one of the following: - `text_embedding` for text embedding model endpoints - `generic` for all other model endpoints |
| `MODEL_QUALIFIED_NAME` | required for text embedding models with built-in support; optional for other model endpoints | The fully qualified name for text embedding models with built-in support. For Vertex AI qualified names that you must use for pre-registered models, see [Pre-registered Vertex AI models](https://docs.cloud.google.com/alloydb/omni/containers/16.8.0/docs/reference/model-endpoint.md.txt#pre-registered-models). For qualified names that you must use for OpenAI models with built-in support, see [Models with built-in support](https://docs.cloud.google.com/alloydb/omni/containers/16.8.0/docs/reference/model-endpoint.md.txt#models-built-in) |
| `AUTH_TYPE` | optional unless the model endpoint has specific authentication requirement | The authentication type used by the model endpoint. You can set it to either `alloydb_service_agent_iam` for Vertex AI models or `secret_manager` for other providers, if they use Secret Manager for authentication. You don't need to set this value if you are using authentication headers. |
| `AUTH_ID` | don't set for Vertex AI model endpoints; required for all other model endpoints that store secrets in Secret Manager | The secret ID that you set and is subsequently used when registering a model endpoint. |
| `GENERATE_HEADER_FUNCTION` | optional | The name of the function that generates custom headers. For Anthropic models, model endpoint management provides a `google_ml.anthropic_claude_header_gen_fn` function that you can use for default versions. The signature of this function depends on the prediction function that you use. See [Header generation function](https://docs.cloud.google.com/alloydb/omni/containers/16.8.0/docs/reference/model-endpoint.md.txt#http_header_generation_function). |
| `INPUT_TRANSFORM_FUNCTION` | optional for text embedding model endpoints with built-in support; don't set for generic model endpoints | The function to transform input of the corresponding prediction function to the model-specific input. See [Transform functions](https://docs.cloud.google.com/alloydb/omni/containers/16.8.0/docs/reference/model-endpoint.md.txt#transform-functions). |
| `OUTPUT_TRANSFORM_FUNCTION` | optional for text embedding model endpoints with built-in support; don't set for generic model endpoints | The function to transform model specific output to the prediction function output. See [Transform functions](https://docs.cloud.google.com/alloydb/omni/containers/16.8.0/docs/reference/model-endpoint.md.txt#transform-functions). |

### `google_ml.alter_model()`

The following shows how to call the `google_ml.alter_model()` SQL function used
to update model endpoint metadata:

```sql
    CALL
    google_ml.alter_model(
      model_id => 'MODEL_ID',
      model_request_url => 'REQUEST_URL',
      model_provider => 'PROVIDER_ID',
      model_type => 'MODEL_TYPE',
      model_qualified_name => 'MODEL_QUALIFIED_NAME',
      model_auth_type => 'AUTH_TYPE',
      model_auth_id => 'AUTH_ID',
      generate_headers_fn => 'GENERATE_HEADER_FUNCTION',
      model_in_transform_fn => 'INPUT_TRANSFORM_FUNCTION',
      model_out_transform_fn => 'OUTPUT_TRANSFORM_FUNCTION');
```

For information about the values that you must set for each parameter, see [Create a model](https://docs.cloud.google.com/alloydb/omni/containers/16.8.0/docs/reference/model-endpoint.md.txt#create-model).

### `google_ml.drop_model()` function

The following shows how to call the `google_ml.drop_model()` SQL function used
to drop a model endpoint:

```sql
  CALL google_ml.drop_model('MODEL_ID');
```

| Parameter | Description |
|---|---|
| `MODEL_ID` | A unique ID for the model endpoint that you defined. |

### `google_ml.list_model()` function

The following shows how to call the `google_ml.list_model()` SQL function used
to list model endpoint information:

```sql
  SELECT google_ml.list_model('MODEL_ID');
```

| Parameter | Description |
|---|---|
| `MODEL_ID` | A unique ID for the model endpoint that you defined. |

### `google_ml.model_info_view` view

The following shows how to call the `google_ml.model_info_view` view that is
used to list model endpoint information for all model endpoints:

```sql
  SELECT * FROM google_ml.model_info_view;
```

## Secrets

Use this reference to understand parameters for functions that let you manage
secrets.

### `google_ml.create_sm_secret()` function

The following shows how to call the `google_ml.create_sm_secret()` SQL function
used to add the secret created in Secret Manager:

```sql
    CALL
    google_ml.create_sm_secret(
      secret_id => 'SECRET_ID',
      secret_path => 'projects/project-id/secrets/SECRET_MANAGER_SECRET_ID/versions/VERSION_NUMBER');
```

| Parameter | Description |
|---|---|
| `SECRET_ID` | The secret ID that you set and is subsequently used when registering a model endpoint. |
| `PROJECT_ID` | The ID of your Google Cloud project that contains the secret. |
| `SECRET_MANAGER_SECRET_ID` | The secret ID set in Secret Manager when you created the secret. |
| `VERSION_NUMBER` | The version number of the secret ID. |

### `google_ml.alter_sm_secret()` function

The following shows how to call the `google_ml.alter_sm_secret()` SQL function
used to update secret information:

```sql
  CALL
    google_ml.alter_sm_secret(
      secret_id => 'SECRET_ID',
      secret_path => 'projects/project-id/secrets/SECRET_MANAGER_SECRET_ID/versions/VERSION_NUMBER');
```

For information about the values that you must set for each parameter, see [Create a secret](https://docs.cloud.google.com/alloydb/omni/containers/16.8.0/docs/reference/model-endpoint.md.txt#create-secret).

### `google_ml.drop_sm_secret()` function

The following shows how to call the `google_ml.drop_sm_secret()` SQL function
used to drop a secret:

```sql
  CALL google_ml.drop_sm_secret('SECRET_ID');
```

| Parameter | Description |
|---|---|
| `SECRET_ID` | The secret ID that you set and was subsequently used when registering a model endpoint. |

## Prediction functions

Use this reference to understand parameters for functions that let you generate
embeddings or invoke predictions.

### `google_ml.embedding()` function

The following shows how to generate embeddings:

```sql
SELECT
  google_ml.embedding(
    model_id => 'MODEL_ID',
    contents => 'CONTENT');
```

| Parameter | Description |
|---|---|
| `MODEL_ID` | A unique ID for the model endpoint that you define. |
| `CONTENT` | The text to translate into a vector embedding. |

For example SQL queries to generate text embeddings, see [Transform function examples for
AlloyDB Omni](https://docs.cloud.google.com/alloydb/omni/containers/16.8.0/docs/ai/model-endpoint-embeddings#generate_embeddings).

### `google_ml.predict_row()` function

The following shows how to invoke predictions:

```sql
SELECT
  google_ml.predict_row(
    model_id => 'MODEL_ID',
    request_body => 'REQUEST_BODY');
```

| Parameter | Description |
|---|---|
| `MODEL_ID` | A unique ID for the model endpoint that you define. |
| `REQUEST_BODY` | The parameters to the prediction function, in JSON format. |

For example SQL queries to invoke predictions, see [Examples for AlloyDB Omni](https://docs.cloud.google.com/alloydb/omni/containers/16.8.0/docs/ai/model-endpoint-predictions.md.txt#examples).

## Transform functions

Use this reference to understand parameters for input and output transform
functions.

### Input transform function

The following shows the signature for the prediction function for text embedding
model endpoints:

```sql
  CREATE OR REPLACE FUNCTION INPUT_TRANSFORM_FUNCTION(model_id VARCHAR(100), input_text TEXT) RETURNS JSON;
```

| Parameter | Description |
|---|---|
| `INPUT_TRANSFORM_FUNCTION` | The function to transform input of the corresponding prediction function to the model endpoint-specific input. |

### Output transform function

The following shows the signature for the prediction function for text embedding
model endpoints:

```sql
  CREATE OR REPLACE FUNCTION OUTPUT_TRANSFORM_FUNCTION(model_id VARCHAR(100), response_json JSON) RETURNS real[];
```

| Parameter | Description |
|---|---|
| `OUTPUT_TRANSFORM_FUNCTION` | The function to transform model endpoint-specific output to the prediction function output. |

### Transform functions example

To better understand how to create transform functions for your model endpoint,
consider a custom-hosted text embedding model endpoint that requires JSON input
and output.

The following example cURL request creates embeddings based on the prompt and
the model endpoint:

```bash
  curl -m 100 -X POST https://cymbal.com/models/text/embeddings/v1 \
    -H "Content-Type: application/json" 
    -d '{"prompt": ["AlloyDB Embeddings"]}'
```

The following example response is returned:

```json
[[ 0.3522231  -0.35932037  0.10156056  0.17734447 -0.11606089 -0.17266059
   0.02509351  0.20305622 -0.09787305 -0.12154685 -0.17313677 -0.08075467
   0.06821183 -0.06896557  0.1171584  -0.00931572  0.11875633 -0.00077482
   0.25604948  0.0519384   0.2034983  -0.09952664  0.10347155 -0.11935943
  -0.17872004 -0.08706985 -0.07056875 -0.05929353  0.4177883  -0.14381726
   0.07934926  0.31368294  0.12543282  0.10758053 -0.30210832 -0.02951015
   0.3908268  -0.03091059  0.05302926 -0.00114946 -0.16233777  0.1117468
  -0.1315904   0.13947351 -0.29569918 -0.12330773 -0.04354299 -0.18068913
   0.14445548  0.19481727]]
```

Based on this input and response, we can infer the following:

- The model expects JSON input through the `prompt` field. This field accepts an
  array of inputs. As the `google_ml.embedding()` function is a row level
  function, it expects one text input at a time. Thus,you need to create an
  input transform function that builds an array with single element.

- The response from the model is an array of embeddings, one for each prompt
  input to the model. As the `google_ml.embedding()` function is a row level
  function, it returns single input at a time. Thus, you need to create an
  output transform function that can be used to extract the embedding from the
  array.

The following example shows the input and output transform functions that is
used for this model endpoint when it is registered with model endpoint management:

### input transform function

```sql
CREATE OR REPLACE FUNCTION cymbal_text_input_transform(model_id VARCHAR(100), input_text TEXT)
RETURNS JSON
LANGUAGE plpgsql
AS $$
DECLARE
  transformed_input JSON;
  model_qualified_name TEXT;
BEGIN
  SELECT json_build_object('prompt', json_build_array(input_text))::JSON INTO transformed_input;
  RETURN transformed_input;
END;
$$;
```

### output transform function

```sql
CREATE OR REPLACE FUNCTION cymbal_text_output_transform(model_id VARCHAR(100), response_json JSON)
RETURNS REAL[]
LANGUAGE plpgsql
AS $$
DECLARE
  transformed_output REAL[];
BEGIN
SELECT ARRAY(SELECT json_array_elements_text(response_json->0)) INTO transformed_output;
RETURN transformed_output;
END;
$$;
```

## HTTP header generation function

The following shows signature for the header generation function that can be
used with the `google_ml.embedding()` prediction function when registering other
text embedding model endpoints.

```sql
  CREATE OR REPLACE FUNCTION GENERATE_HEADERS(model_id VARCHAR(100), input_text TEXT) RETURNS JSON;
```

For the `google_ml.predict_row()` prediction function, the signature is as
follows:

```sql
CREATE OR REPLACE FUNCTION GENERATE_HEADERS(model_id TEXT, input JSON) RETURNS JSON;
```

| Parameter | Description |
|---|---|
| `GENERATE_HEADERS` | The function to generate custom headers. You can also pass the authorization header generated by the header generation function while registering the model endpoint. |

### Header generation function example

To better understand how to create a function that generates output in JSON key
value pairs that are used as HTTP headers, consider a custom-hosted text
embedding model endpoint.

The following example cURL request passes the `version` HTTP header which is
used by the model endpoint:

```bash
  curl -m 100 -X POST https://cymbal.com/models/text/embeddings/v1 \
      -H "Content-Type: application/json" \
      -H "version: 2024-01-01" \
      -d '{"prompt": ["AlloyDB Embeddings"]}'
```

The model expects text input through the `version` field and returns the version
value in JSON format. The following example shows the header generation function
that is used for this text embedding model endpoint when it is registered with model
endpoint management:

```sql
CREATE OR REPLACE FUNCTION header_gen_fn(model_id VARCHAR(100), input_text TEXT)
RETURNS JSON
LANGUAGE plpgsql
AS $$
BEGIN
  RETURN json_build_object('version', '2024-01-01')::JSON;
END;
$$;
```

#### Header generation function using API Key

The following examples show how to set up authentication using the API key.

### embedding model

```sql
CREATE OR REPLACE FUNCTION header_gen_func(
  model_id VARCHAR(100),
  input_text TEXT
)
RETURNS JSON
LANGUAGE plpgsql
AS $$
#variable_conflict use_variable
BEGIN
  RETURN json_build_object('Authorization', 'API_KEY')::JSON;
END;
$$;
```

Replace the `API_KEY` with the API key of the model provider.

### generic model

```sql
CREATE OR REPLACE FUNCTION header_gen_func(
  model_id VARCHAR(100),
  response_json JSON
)
RETURNS JSON
LANGUAGE plpgsql
AS $$
#variable_conflict use_variable
DECLARE
transformed_output REAL[];
BEGIN
  -- code to add Auth token to API request
RETURN json_build_object('x-api-key', 'API_KEY', 'anthropic-version', '2023-06-01')::JSON;
END;
$$;
```

Replace the `API_KEY` with the API key of the model provider.

## Request URL generation

Use the request URL generation function to infer the request URLs for the model
endpoints with built-in support. The following shows the signature for this
function:

```sql
CREATE OR REPLACE FUNCTION GENERATE_REQUEST_URL(provider google_ml.model_provider, model_type google_ml.MODEL_TYPE, model_qualified_name VARCHAR(100), model_region VARCHAR(100) DEFAULT NULL)
```

| Parameter | Description |
|---|---|
| `GENERATE_REQUEST_URL` | The function to generate request URL generated by the extension for model endpoints with built-in support. |

## Supported models

You can use model endpoint management to register any text embedding or generic model endpoint. Model endpoint management also includes pre-registered Vertex AI models and models with built-in support. For more information about different model types, see [Model type](https://docs.cloud.google.com/alloydb/omni/containers/16.8.0/docs/ai/model-endpoint-overview.md.txt).

### Pre-registered Vertex AI models

> [!NOTE]
> **Note:** Vertex AI model support is governed by Vertex AI model versioning and lifecycle guidelines. For more information about stable versions, see [Model versions and lifecycle](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versions).

| Model type | Model ID | Extension version |
|---|---|---|
| `generic` | - `gemini-1.5-pro:streamGenerateContent` - `gemini-1.5-pro:generateContent` - `gemini-1.0-pro:generateContent` | version 1.4.2 and later |
| `text_embedding` | - `textembedding-gecko` - `text-embedding-gecko@001` | version 1.3 and later |

### Models with built-in support

### Vertex AI

| Qualified model name | Model type |
|---|---|
| `text-embedding-gecko@001` | `text-embedding` |
| `text-embedding-gecko@003` | `text-embedding` |
| `text-embedding-004` | `text-embedding` |
| `text-embedding-005` | `text-embedding` |
| `text-embedding-preview-0815` | `text-embedding` |
| `text-multilingual-embedding-002` | `text-embedding` |

### OpenAI

| Qualified model name | Model type |
|---|---|
| `text-embedding-ada-002` | `text-embedding` |
| `text-embedding-3-small` | `text-embedding` |
| `text-embedding-3-large` | `text-embedding` |

### Anthropic

| Qualified model name | Model type |
|---|---|
| `claude-3-opus-20240229` | `generic` |
| `claude-3-sonnet-20240229` | `generic` |
| `claude-3-haiku-20240307` | `generic` |