Integrate Cloud SQL with Vertex AI

This page describes how to integrate Cloud SQL with Vertex AI.

This integration lets you apply large language models (LLMs), which are hosted in Vertex AI, to a Cloud SQL for SQL Server 2025 database.

By integrating Cloud SQL with Vertex AI, you can apply the semantic and predictive power of machine learning (ML) models to your data. This integration extends the SQL syntax with the generate embeddings function for querying your models.

You use the generate embeddings function so that the embedding model can translate text prompts into numerical vectors. You can then apply these vector embeddings as inputs to vector data types. For more information, see the Microsoft documentation on Vector search and vector indexes in the SQL Database Engine.

For more information about Vertex AI, see Introduction to Vertex AI.

Supported editions and database versions

Vertex AI integration is supported for both Cloud SQL Enterprise Plus edition and Cloud SQL Enterprise edition editions and for all database versions of SQL Server 2025.

Supported model types

Vertex AI integration with Cloud SQL supports model endpoints from the following sources:

  • Vertex AI
  • Hugging Face
  • OpenAI

Before you begin

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

    Go to project selector

  3. Verify that billing is enabled for your Google Cloud project.

  4. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

    Go to project selector

  5. Verify that billing is enabled for your Google Cloud project.

  6. Enable the necessary Google Cloud APIs.

    Console

    1. Go to the APIs & Services page.
    2. From the projects list, select your project.
    3. If the API Library isn't open, then from the navigation menu, select Library.

    4. Click the APIs that you want to enable. For this procedure, enable the following:

      • Cloud SQL Admin API
      • Vertex AI API
      • Compute Engine API
      • Optional: If using a third-party model, then the Secret Manager API is required.
    5. After selecting each API, click Enable.

    gcloud

    1. Open Cloud Shell, which provides command-line access to your Google Cloud resources directly from the browser.
    2. To enable the required APIs, use the gcloud services enable command: 
      gcloud services enable sqladmin.googleapis.com \
  aiplatform.googleapis.com \
  compute.googleapis.com \
  secretmanager.googleapis.com
      3. This command enables the following APIs:
      • Cloud SQL Admin API
      • Vertex AI API
      • Compute Engine API
      • Optional: If using a third-party model, then the Secret Manager API is required.

  7. Grant the Cloud SQL service account Identity and Access Management (IAM) permissions to access Vertex AI.

    8. gcloud

    To add the Vertex AI permissions to the Cloud SQL service account for the project where the Cloud SQL instance is located, use the gcloud projects add-iam-policy-binding command: 

    gcloud projects add-iam-policy-binding PROJECT_ID \
--member="serviceAccount:SERVICE_ACCOUNT_EMAIL" \
--role="roles/aiplatform.user"

    Make the following replacements:
    • PROJECT_ID: the ID of the project that has the Vertex AI endpoint. Cloud SQL uses this endpoint to access the LLM that's hosted in Vertex AI.

    • SERVICE_ACCOUNT_EMAIL: the email address of the Cloud SQL service account.

      To find this email address, use the gcloud sql instances describe command: 

      gcloud sql instances describe INSTANCE_NAME | grep EmailAddress

      Replace INSTANCE_NAME with the name of the Cloud SQL instance.

  8. Make sure the Cloud SQL user who is working with the embedding models is granted the following database permissions:
    • To let a database user create or modify models, you need to grant the Cloud SQL user the EXECUTE permission on the following stored procedures:
      • [msdb].gcloudsql_ml_create_external_model: creates an external model object that contains the location, authentication method, and purpose of an AI model inference endpoint.
      • [msdb].gcloudsql_ml_alter_external_model: lets you modify the external model object.
      • [msdb].gcloudsql_ml_drop_external_model: removes the external model object from the database.
    • To call the stored procedures previously listed, you need the following permissions in the target database:
      • CREATE EXTERNAL MODEL
      • ALTER EXTERNAL MODEL

      • To see how to grant these permissions, see Microsoft documentation on the Create external model command and nomenclature.

      For information about Cloud SQL superuser permissions, see About SQL Server users and stored procedures.

  9. If you want to use a third-party external model, then you must have a valid API Key for the model stored in a Secret Manager location accessible to the Cloud SQL instance.
  10. If using a third-party external model, then the Cloud SQL instance must use a public IP address. Private IP instances can't access third-party external models. Private services access and Private Service Connect are only supported with Vertex AI integration.

Enable database integration with Vertex AI

To enable database integration with Vertex AI, complete the following steps:

  1. Create or update a Cloud SQL instance so that the instance can integrate with Vertex AI.

    gcloud

    Create the instance

    To create the Cloud SQL instance, use the gcloud sql instances create command.

    gcloud sql instances create INSTANCE_NAME \
--database-version=DATABASE_VERSION \
--tier=MACHINE_TYPE \
--region=REGION_NAME \
--enable-google-ml-integration

    Make the following replacements:

    • INSTANCE_NAME: the name of the instance.
    • DATABASE_VERSION: the database version for the instance, such as SQLSERVER_2025_ENTERPRISE.
    • MACHINE_TYPE: the machine type for the instance, such as db-perf-optimized-N-8..
    • REGION_NAME: the region for the instance, such as us-central1.

    Update the instance

    To update the instance, use the gcloud sql instances patch command.

    gcloud sql instances patch INSTANCE_NAME \
--enable-google-ml-integration

    REST v1

    Create the instance

    Use this example to create the instance. For a complete list of parameters for this call, see the instances:insert page. For information about instance settings, including valid values for a region, see Instance settings.

    Before using any of the request data, make the following replacements:

    • PROJECT_ID: the ID or project number of the Google Cloud project that contains the instance.
    • INSTANCE_NAME: the name of the instance.
    • REGION_NAME: the region name for the instance.
    • DATABASE_VERSION: enum string of the database version, such as SQLSERVER_2025_ENTERPRISE.
    • PASSWORD: the password for the database root user.
    • MACHINE_TYPE: enum string of the machine (tier) type, such as db-custom-[CPUS]-[MEMORY_MBS]. For more information, see Machine tiers.
    • EDITION_TYPE: your Cloud SQL edition, such as ENTERPRISE.

    You must also include the enableGoogleMlIntegration object in the request. Set the following parameters, as required:

    • enableGoogleMlIntegration: when this parameter is set to true, Cloud SQL instances can connect to Vertex AI to pass requests for real-time predictions and insights to the AI.

    HTTP method and URL: 

    POST https://sqladmin.googleapis.com/v1/projects/PROJECT_ID/instances

    Request JSON body: 

    {
  "name": "INSTANCE_NAME",
  "region": "REGION_NAME",
  "databaseVersion": "DATABASE_VERSION",
  "rootPassword": "PASSWORD",
  "settings": {
    "tier": "MACHINE_TYPE",
    "edition": "EDITION_TYPE",
    "enableGoogleMlIntegration": "true" | "false"
    
  }
}

    To send your request, expand one of these options:

    You should receive a JSON response similar to the following:

    {
  "kind": "sql#operation",
  "targetLink": "https://sqladmin.googleapis.com/v1/projects/PROJECT_ID/instances/INSTANCE_ID",
  "status": "PENDING",
  "user": "user@example.com",
  "insertTime": "2019-09-25T22:19:33.735Z",
  "operationType": "CREATE",
  "name": "OPERATION_ID",
  "targetId": "INSTANCE_ID",
  "selfLink": "https://sqladmin.googleapis.com/v1/projects/PROJECT_ID/operations/OPERATION_ID",
  "targetProject": "PROJECT_ID"
}

    Update the instance

    Use this example to update the instance. For a complete list of parameters for this call, see the instances.patch page.

    If this update modifies a value that requires a restart, then you see a prompt to proceed with the change or cancel.

    Before using any of the request data, make the following replacements:

    • PROJECT_ID: the ID or project number of the Google Cloud project that contains the instance
    • INSTANCE_NAME: the name of the instance

    HTTP method and URL: 

    PATCH https://sqladmin.googleapis.com/v1/projects/PROJECT_ID/instances/INSTANCE_NAME

    Request JSON body: 

    {
  "settings": {
    "enableGoogleMlIntegration": true,
    
   }
}

    To send your request, expand one of these options:

    You should receive a JSON response similar to the following:

    {
  "kind": "sql#operation",
  "targetLink": "https://sqladmin.googleapis.com/v1/projects/PROJECT_ID/instances/INSTANCE_NAME",
  "status": "PENDING",
  "user": "user@example.com",
  "insertTime": "2020-01-16T02:32:12.281Z",
  "operationType": "UPDATE",
  "name": "OPERATION_ID",
  "targetId": "INSTANCE_NAME",
  "selfLink": "https://sqladmin.googleapis.com/v1/projects/PROJECT_ID/operations/OPERATION_ID",
  "targetProject": "PROJECT_ID"
}

    REST v1beta4

    Create the instance

    Use this example to create the instance. For a complete list of parameters for this call, see the instances:insert page. For information about instance settings, including valid values for a region, see Instance settings.

    Before using any of the request data, make the following replacements:

    • PROJECT_ID: the ID or project number of the Google Cloud project that contains the instance.
    • INSTANCE_NAME: the name of the instance.
    • REGION_NAME: the region name for the instance.
    • DATABASE_VERSION: enum string of the database version, such as SQLSERVER_2025_ENTERPRISE.
    • PASSWORD: the password for the database root user.
    • MACHINE_TYPE: enum string of the machine (tier) type, such as db-custom-[CPUS]-[MEMORY_MBS]. For more information, see Machine tiers.
    • EDITION_TYPE: your Cloud SQL edition, such as ENTERPRISE.

    You must also include the enableGoogleMlIntegration object in the request. Set the following parameters, as required:

    • enableGoogleMlIntegration: when this parameter is set to true, Cloud SQL instances can connect to Vertex AI to pass requests for real-time predictions and insights to the AI.

    HTTP method and URL: 

    POST https://sqladmin.googleapis.com/v1beta4/projects/PROJECT_ID/instances

    Request JSON body: 

    {
  "name": "INSTANCE_NAME",
  "region": "REGION_NAME",
  "databaseVersion": "DATABASE_VERSION",
  "rootPassword": "PASSWORD",
  "settings": {
    "tier": "MACHINE_TYPE",
    "edition": "EDITION_TYPE",
    "enableGoogleMlIntegration": "true" | "false"
    
  }
}

    To send your request, expand one of these options:

    You should receive a JSON response similar to the following:

    {
  "kind": "sql#operation",
  "targetLink": "https://sqladmin.googleapis.com/v1beta4/projects/PROJECT_ID/instances/INSTANCE_ID",
  "status": "PENDING",
  "user": "user@example.com",
  "insertTime": "2019-09-25T22:19:33.735Z",
  "operationType": "CREATE",
  "name": "OPERATION_ID",
  "targetId": "INSTANCE_ID",
  "selfLink": "https://sqladmin.googleapis.com/v1beta4/projects/PROJECT_ID/operations/OPERATION_ID",
  "targetProject": "PROJECT_ID"
}

    Update the instance

    Use this example to update the instance. For a complete list of parameters for this call, see the instances.patch page.

    If this update modifies a value that requires a restart, then you see a prompt to proceed with the change or cancel.

    Before using any of the request data, make the following replacements:

    • PROJECT_ID: the ID or project number of the Google Cloud project that contains the instance
    • INSTANCE_NAME: the name of the instance

    HTTP method and URL: 

    PATCH https://sqladmin.googleapis.com/v1beta4/projects/PROJECT_ID/instances/INSTANCE_NAME

    Request JSON body: 

    {
  "settings": {
    "enableGoogleMlIntegration": true,
    
   }
}

    To send your request, expand one of these options:

    You should receive a JSON response similar to the following:

    {
  "kind": "sql#operation",
  "targetLink": "https://sqladmin.googleapis.com/v1beta4/projects/PROJECT_ID/instances/INSTANCE_NAME",
  "status": "PENDING",
  "user": "user@example.com",
  "insertTime": "2020-01-16T02:32:12.281Z",
  "operationType": "UPDATE",
  "name": "OPERATION_ID",
  "targetId": "INSTANCE_NAME",
  "selfLink": "https://sqladmin.googleapis.com/v1beta4/projects/PROJECT_ID/operations/OPERATION_ID",
  "targetProject": "PROJECT_ID"
}

Add Secret Manager permissions

If you want to use a model from an external source, such as OpenAI or Hugging Face, then you must store the model source's API key in Secret Manager.

Then, you need to give the Cloud SQL instance access to the model source's API key in Secret Manager. To gain access, run the following command:


SA_NAME=$(gcloud sql instances describe INSTANCE_NAME --format="value(serviceAccountEmailAddress)");

gcloud secrets add-iam-policy-binding SECRET_NAME \
    --member="serviceAccount:${SA_NAME}" \
    --role="roles/secretmanager.secretAccessor"

Replace the following:

  • INSTANCE_NAME: the name of the Cloud SQL instance.
  • SECRET_NAME: the secret's name as used in Secret Manager.

External model objects

To access a model from an external source, you need to create a model object for SQL Server to use. Cloud SQL provides three stored procedures that let you create, alter, and drop an external model object:

  • gcloudsql_ml_create_external_model
  • gcloudsql_ml_alter_external_model
  • gcloudsql_ml_drop_external_model

Create an external model

To create a model, run the gcloudsql_ml_create_external_model stored procedure found in the msdb database:

EXECUTE [msdb].[dbo].[gcloudsql_ml_create_external_model]
@db = [DB_NAME],
@model_name = MODEL_NAME,
@model_provider = 'MODEL_PROVIDER',
@model = 'MODEL',
@model_url = MODEL_URL,
@secret_url = SECRET_URL

Replace the following:

  • DB_NAME: the target database where you want the external model to be created.
  • MODEL_NAME: the name of the new external model.
  • MODEL_PROVIDER: the name of the model provider, such as one of the following:
    • Vertex AI
    • OpenAI
    • Hugging Face
  • MODEL_URL: the URL of the model endpoint.
  • MODEL: the AI model being invoked. For example, gemini-embedding-002.
  • SECRET_URL: if Vertex AI is the model provider, this parameter must be empty. If using an external model, this value must reference the Secret Manager location of the API Key. An example URL uses the following format: 
        https://secretmanager.googleapis.com/v1/projects/PROJECT_ID/secrets/SECRET_NAME/versions/VERSION

    Replace the following:

    • PROJECT_ID: the project ID where the secret is located.
    • SECRET_NAME: the name of the secret as used in Secret Manager.
    • VERSION: the version number of the secret.

Alter an external model

To alter an external model object, run the gcloudsql_ml_alter_external_model stored procedure found in the msdb database:

EXECUTE [msdb].[dbo].[gcloudsql_ml_alter_external_model]
@db = [DB_NAME],
@model_name = MODEL_NAME,
@model_provider = 'MODEL_PROVIDER',
@model = 'MODEL',
@model_url = MODEL_URL,
@secret_url = SECRET_URL

Replace the following:

  • DB_NAME: the target database where the model you want to modify exists.
  • MODEL_NAME: the name of the existing model.
  • MODEL_PROVIDER: the name of the model provider, such as one of the following:
    • Vertex AI
    • OpenAI
    • Hugging Face
  • MODEL_URL: the URL of the model endpoint.
  • MODEL: the AI model being invoked. For example, gemini-embedding-001.
  • SECRET_URL: if Vertex AI is the model provider, this parameter must be empty. If using an external model, this value must reference the Secret Manager location of the API Key. An example URL uses the following format: 
        https://secretmanager.googleapis.com/v1/projects/PROJECT_ID/secrets/SECRET_NAME/versions/VERSION

    Replace the following:

    • PROJECT_ID: the project ID where the secret is located.
    • SECRET_NAME: the name of the secret as used in Secret Manager.
    • VERSION: the version number of the secret.

Disable Vertex AI integration

To disable Vertex AI integration on a Cloud SQL instance, run the following:

gcloud sql instances patch [instance-name] --no-enable-google-ml-integration

Replace INSTANCE_NAME with the name of the Cloud SQL instance.

Drop an external model in SQL Server

To drop an external model from SQL Server, run the gcloudsql_ml_drop_external_model stored procedure found in the msdb database:

EXECUTE [msdb].[dbo].[gcloudsql_ml_drop_external_model]
@db = [DB_NAME],
@model_name = MODEL_NAME

Replace the following:

  • DB_NAME: the target database where the external model you want to drop exists.
  • MODEL_NAME: the name of the external model to drop.

Invoke an external model

After Vertex AI integration is enabled on the Cloud SQL instance, and a model object is created for SQL Server to use, you can invoke the SQL Server AI_GENERATE_EMBEDDINGS function to create vector embeddings for your data.

To create vector embeddings using the external model, run the following command:

SELECT AI_GENERATE_EMBEDDINGS(N'Test Text' USE MODEL MODEL_NAME)

Replace MODEL_NAME with the name of the model you want to use.

The executing principal user must have been granted EXECUTE ON EXTERNAL MODEL permissions using an appropriate role or grant.

For more information, see the Microsoft documentation on External model grants.

Model size restrictions

Some models, such as gemini-embedding-01, have higher default output dimensions than SQL Server can support for float32 vectors, which have a maximum of 1,998 dimensions. When necessary, you must either use float16 (a Microsoft preview feature) for the vector base type, which offers a maximum of 3,996 vector dimensions, or the dimensions parameter must be provided in the AI_GENERATE_EMBEDDINGS request to reduce the dimensionality.

For more information, see Half-precision float support in vector data type.

For an example, see the following:

DECLARE @params JSON = N'{"dimensions": "DIMENSIONS"}';
SELECT AI_GENERATE_EMBEDDINGS('This article introduces AI concepts.' USE MODEL MODEL_NAME PARAMETERS @params)

Replace the following:

  • DIMENSIONS: the dimensions you want to use for your model, such as 1536. This value is accepted as a string.
  • MODEL_NAME: the name of the model you want to use.

Troubleshoot

The following are potential error messages you might receive:

Context Error or message received Possible cause
CREATE/ALTER procedure "The specified model provider is not valid." An unsupported model provider was given.
CREATE/ALTER procedure "The specified secret URL is not valid for this model provider." For Vertex AI, the secret URL must be empty. For other model providers, the secret URL must not be empty.
CREATE/ALTER/DROP procedure "The specified model name is not valid." Model name is a required field.
CREATE/ALTER procedure "The specified model URL is not valid." Model URL is a required field.
CREATE/ALTER/DROP procedure "The specified DB name is not valid." A valid DB name is a required field.
AI_GENERATE_EMBEDDINGS call Various error codes, including 400, 403, 404, 405, 500. If running into issues when attempting to run the AI_GENERATE_EMBEDDINGS function, only error codes are returned. This is a known Microsoft limitation.
AI_GENERATE_EMBEDDINGS call "The HTTP response does not contain valid JSON." An error code that asks you to try again, such as 429 or 500, was returned from the machine learning agent. However, the client receives this text message instead. This is a known Microsoft limitation.

Error codes

The AI_GENERATE_EMBEDDINGS function only provides error codes, not messages. The following table lists possible causes for some error codes:

Error code Possible cause
400 Bad Request
  • Unsupported model type.
  • Incorrectly-formed JSON payload.
  • Invalid URL for the model type.
401 Unauthorized The third-party model lacks a valid API key.
403 Forbidden
  • The Cloud SQL instance lacks appropriate permissions to access Vertex AI.
  • The Cloud SQL instance lacks appropriate permissions to access Secret Manager.
  • The API key does not grant permission to the specified third-party model.
  • Vertex AI is not enabled on the project.
404 Not Found
  • The specified secret_url does not exist.
  • TThe specified model_url does not exist.
405 Method Not Allowed A method other than POST was provided.
413 Request Body Too Large The request size is larger than the 1MB limit.
429 Too Many Requests Exhausted error from the model provider.
500 Internal Server Error
  • Unspecified errors from the model provider.
  • Unexpected reply from the model provider.

What's next