For AutoML tabular models, AutoML image models,
and custom-trained models, you can enable or disable inference logs during
model deployment or endpoint
creation. This page explains the different types of inference
logs available, and how to enable or disable these logs.

## Types of inference logs

There are several types of inference logs that you can use to get information
from your inference nodes:

- **Container logging** , which logs the `stdout` and `stderr` streams from your
  inference nodes to [Cloud Logging](https://docs.cloud.google.com/logging/docs/overview.md.txt). These logs
  are required for debugging.

  - On the `v1` service endpoint, container logging is enabled by default.
    You can disable it when you deploy a model. You can also disable or
    enable logging when you [mutate](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.endpoints/mutateDeployedModel.md.txt)
    the deployed model.

  - On the `v1beta1` service endpoint, container logging is disabled by
    default. You can enable it when you deploy a model. You can also disable
    or enable logging when you [mutate](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.endpoints/mutateDeployedModel.md.txt)
    the deployed model.

  > [!NOTE]
  > **Note:** The default logging behavior in Python sends outputs to `stderr`, which will appear at the `ERROR` level in Cloud Logging. If you'd like for container logs to appear at the `INFO` level, configure your container logging to send outputs to `stdout`. For more information, see the Python [Logging handlers](https://docs.python.org/3/library/logging.handlers.html) tutorials and the Python [Logging Cookbook](https://docs.python.org/3/howto/logging-cookbook.html).

- **Access logging**, which logs information like timestamp and latency for
  each request to Cloud Logging.

  On both the `v1` and `v1beta1` service endpoints, access logging is disabled
  by default. You can enable access logging when you deploy a model to an
  endpoint.
- **Request-response logging**, which logs a sample of online inference
  requests and responses to a BigQuery table.

  You can enable request-response logging by creating or patching the
  inference endpoint.

You can enable or disable each type of log independently.

## Inference log settings

You can enable or disable online inference logs when you create an endpoint,
deploy a model to the endpoint, or mutate a deployed model.

To update the settings for access logs, you must
[undeploy your model](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.endpoints/undeployModel.md.txt),
and then redeploy the model with your new settings. You can update the settings
for container logs without re-deploying your model.

Online inference at a high rate of queries per second (QPS) can produce a
substantial number of logs, which are subject to
[Cloud Logging pricing](https://cloud.google.com/stackdriver/pricing#logging-costs). To estimate the pricing for
your online inference logs, see
[Estimating your bills](https://docs.cloud.google.com/stackdriver/estimating-bills.md.txt) for logging. To reduce
this cost, you can disable inference logging.

### Enable and disable inference logs

The following examples highlight where to modify the default log settings:

### Console

When you deploy a model to an endpoint or create a new endpoint in the
Google Cloud console, you can specify which types of inference logs to enable
in the **Logging** step. Select the checkboxes to enable **Access logging** or
**Container logging**, or clear the checkboxes to disable these logs.

Use the REST API to update the settings for container logs.

Use the REST API to enable request-response logging. The
Google Cloud console and gcloud CLI don't support request-response
logging configuration.

To see more context about how to deploy models, read
[Deploy a model using the Google Cloud console](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/predictions/deploy-model-console.md.txt).

### gcloud

To change the default behavior for which logs are enabled in
deployed models, add flags to your `gcloud` command:

### `v1` service endpoint

Run [`gcloud ai endpoints deploy-model`](https://docs.cloud.google.com/sdk/gcloud/reference/ai/endpoints/deploy-model.md.txt):

```bash
gcloud ai endpoints deploy-model ENDPOINT_ID\
  --region=LOCATION \
  --model=MODEL_ID \
  --display-name=DEPLOYED_MODEL_NAME \
  --machine-type=MACHINE_TYPE \
  --accelerator=count=2,type=nvidia-tesla-t4 \
  --disable-container-logging \\
  --enable-access-logging
```

### `v1beta1` service endpoint

Run [`gcloud beta ai endpoints deploy-model`](https://docs.cloud.google.com/sdk/gcloud/reference/beta/ai/endpoints/deploy-model.md.txt):

```bash
gcloud beta ai endpoints deploy-model ENDPOINT_ID\
  --region=LOCATION \
  --model=MODEL_ID \
  --display-name=DEPLOYED_MODEL_NAME \
  --machine-type=MACHINE_TYPE \
  --accelerator=count=2,type=nvidia-tesla-t4 \
  --enable-access-logging \\
  --enable-container-logging
```

Use the REST API to update the settings for container logs.

Use the REST API to enable request-response logging. The
Google Cloud console and gcloud CLI don't support request-response
logging configuration.

To see more context about how to deploy models, read
[Deploy a model using the Agent Platform API](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/predictions/deploy-model-api.md.txt).

### REST

To change the default behavior for which logs are enabled in
deployed models, set the relevant fields to `True`:

### `v1` service endpoint

To disable **container logging** , set the `disableContainerLogging` field to
`True` when you call either
[`projects.locations.endpoints.deployModel`](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.endpoints/deployModel.md.txt) or [`projects.locations.endpoints.mutateDeployedModel`](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.endpoints/mutateDeployedModel.md.txt).

To enable **access logging** , set `enableAccessLogging` to `True`
when deploying your model with
[`projects.locations.endpoints.deployModel`](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.endpoints/deployModel.md.txt).

### `v1beta1` service endpoint

To enable **container logging** , set the `enableContainerLogging` field to
`True` when you call either
[`projects.locations.endpoints.deployModel`](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.endpoints/deployModel.md.txt) or [`projects.locations.endpoints.mutateDeployedModel`](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.endpoints/mutateDeployedModel.md.txt).

To enable **access logging** , set `enableAccessLogging` to `True`
when deploying your model with
[`projects.locations.endpoints.deployModel`](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1beta1/projects.locations.endpoints/deployModel.md.txt).

To see more context about how to deploy models, read
[Deploy a model using the Agent Platform API](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/predictions/deploy-model-api.md.txt).

**Request-response logging**

You can only enable request-response logging when you create an
endpoint using
[`projects.locations.endpoints.create`](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.endpoints/create.md.txt) or patch an existing
endpoint using [`projects.locations.endpoints.patch`](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.endpoints/patch.md.txt).

Request-response logging is done at the endpoint level, so requests
sent to any deployed models under the same endpoint are logged.

When you create or patch an endpoint, populate the
`predictRequestResponseLoggingConfig` field of the [Endpoint resource](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.endpoints.md.txt)
with the following entries:

- `enabled`: set to `True` to enable request-response logging.

- `samplingPercentage`: a number between 0 or 1 defining the fraction of
  requests to log. For example, set this value to `1` in order to log all
  requests or to `0.1` to log 10% of requests.

- [`BigQueryDestination`](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/BigQueryDestination.md.txt): the BigQuery table to be used
  for logging. If you only specify a project name, a new dataset is created with
  the name
  `logging_ENDPOINT_DISPLAY_NAME_ENDPOINT_ID`,
  where `ENDPOINT_DISPLAY_NAME` follows the
  [BigQuery naming rules](https://docs.cloud.google.com/bigquery/docs/datasets.md.txt#dataset-naming). If you don't specify a table
  name, a new table is created with the name `request_response_logging`.

  The schema for the BigQuery table should look like the following:

  | Field name | Type | Mode |
  |---|---|---|
  | `endpoint` | STRING | NULLABLE |
  | `deployed_model_id` | STRING | NULLABLE |
  | `logging_time` | TIMESTAMP | NULLABLE |
  | `request_id` | NUMERIC | NULLABLE |
  | `request_payload` | STRING | REPEATED |
  | `response_payload` | STRING | REPEATED |

The following is an example configuration:

```
{
   "predict_request_response_logging_config": {
     "enabled": true,
     "sampling_rate": 0.5,
     "bigquery_destination": {
       "output_uri": "bq://PROJECT_ID.DATASET_NAME.TABLE_NAME"
     }
   }
}
```

<br />

## Inference request-response logging for dedicated endpoints and Private Service Connect endpoints

For [dedicated endpoints](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/general/deployment.md.txt#create-dedicated-endpoint)
and [Private Service Connect](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/predictions/private-service-connect.md.txt)
endpoints, you can use request-response logging to record request
and response payloads under 10 MB (larger payloads are skipped automatically)
for TensorFlow, PyTorch, sklearn, and XGBoost models.

Request-response logging is available only for the
[`predict`](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.endpoints/predict.md.txt)
and [`rawPredict`](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.endpoints/rawPredict.md.txt)
methods.

To enable request-response logging, populate the
[`predictRequestResponseLoggingConfig`](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.endpoints.md.txt#PredictRequestResponseLoggingConfig)
field of the [Endpoint resource](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.endpoints.md.txt) with the following entries:

- `enabled`: set to `True` to enable request-response logging.

- `samplingRate`: the fraction of requests and responses to log. Set to a number
  that is greater than 0 and less than or equal to 1. For example, set this
  value to `1` in order to log all requests or to `0.1` to log 10% of requests.

- [`BigQueryDestination`](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/BigQueryDestination.md.txt): the BigQuery location for the
  output content, as a URI to a project or table.

The following is an example configuration for creating a dedicated endpoint with
request-response logging enabled:

```bash
curl -X POST \
-H "Content-Type: application/json" \
-H "Authorization: Bearer `gcloud auth print-access-token`" https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/endpoints \
-d '{displayName: "ENDPOINT_NAME", \
     dedicatedEndpointEnabled: true, \
     predictRequestResponseLoggingConfig: { \
       enabled: true, \
       samplingRate: 1.0, \
       bigqueryDestination: { \
          outputUri:"bq://PROJECT_ID" \
       } \
     } \
   }'
```

Replace the following:

- <var translate="no">LOCATION_ID</var>: The region where you are using Agent Platform.
- <var translate="no">PROJECT_NUMBER</var>: The project number for your Google Cloud project.
- <var translate="no">ENDPOINT_NAME</var>: The display name for the endpoint.
- <var translate="no">PROJECT_ID</var>: The project ID for your Google Cloud project.

The following is an example configuration for creating a
Private Service Connect endpoint with request-response logging enabled:

```bash
curl -X POST \
-H "Content-Type: application/json" \
-H "Authorization: Bearer `gcloud auth print-access-token`" https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/endpoints \
-d '{displayName: "ENDPOINT_NAME", \
     privateServiceConnectConfig: { \
       enablePrivateServiceConnect: true, \
       projectAllowlist: ["ALLOWED_PROJECTS"] \
     }, \
     predictRequestResponseLoggingConfig: { \
       enabled: true, \
       samplingRate: 1.0, \
       bigqueryDestination: { \
          outputUri:"bq://PROJECT_ID" \
       } \
     } \
   }'
```

Replace the following:

- <var translate="no">`ALLOWED_PROJECTS`</var>: a comma-separated list of Google Cloud project IDs, each enclosed in quotation marks. For example, `["PROJECTID1", "PROJECTID2"]`. If a project isn't contained in this list, you won't be able to send inference requests to the Agent Platform endpoint from it. Make sure to include <var translate="no">VERTEX_AI_PROJECT_ID</var> in this list so that you can call the endpoint from the same project it's in.

## Request-response logging and Model Monitoring v1

Request-response logging and [Model Monitoring v1](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/model-monitoring/overview.md.txt) use
the same BigQuery table on the backend to log incoming requests.
To prevent unexpected changes to this BigQuery table, the
following limitations are enforced when using both features at the same time:

- If an endpoint has Model Monitoring enabled, you can't
  enable request-response logging for the same endpoint.

- If you enable request-response logging and then
  Model Monitoring on the same endpoint, you won't be able
  to change the request-response logging configuration.

## What's next

- [Estimate pricing](https://docs.cloud.google.com/stackdriver/estimating-bills.md.txt) for online inference logging.
- Deploy a model [using the Google Cloud console](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/predictions/deploy-model-console.md.txt) or [using the Agent Platform API](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/predictions/deploy-model-api.md.txt).
- Learn [how to create a BigQuery table](https://docs.cloud.google.com/bigquery/docs/tables.md.txt).