Configure conversation logging

The Conversational Analytics API lets you turn conversation logging on or off for a data agent and choose the BigQuery table that stores its logs. To review users' questions and the agent's answers, query that table.

Before you begin

Before you configure conversation logging for a data agent, complete the following tasks:

  1. In your Google Cloud project, enable the required APIs for the Conversational Analytics API and BigQuery.
  2. Verify that you have the required Identity and Access Management (IAM) roles.
  3. Make sure that you have a data agent whose data sources reference BigQuery tables. For agent configurations that aren't logged, see Limitations.
  4. Plan a BigQuery dataset in the data agent's project that holds only the logs. All data agents in the project share one service agent, and a dedicated dataset limits its write access to your logs.
  5. Decide who can read the logs. The logs contain your users' full questions and the agent's answers. If users chat with the agent through your own application, you're responsible for any notices, disclosures, or consents that apply to the logged conversations. For more information, see Privacy, data governance, and security.

Required roles

To get the permissions that you need to configure conversation logging for a data agent and query the logs, ask your administrator to grant you the following IAM roles:

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

If you create the log dataset, you're automatically granted the BigQuery Data Owner role on it. If you created the data agent, you're automatically granted the Gemini Data Analytics Data Agent Owner role (roles/geminidataanalytics.dataAgentOwner) on it, which includes permission to change its conversation logging settings.

Users who chat with the data agent don't need additional roles for conversation logging. If conversations aren't logged for some users, see Troubleshoot conversation logging.

Service agent role

The Conversational Analytics API service agent (service-PROJECT_NUMBER@gcp-sa-geminidataanalytics.iam.gserviceaccount.com) writes the logs. Replace PROJECT_NUMBER with the number of the project that contains the data agent.

To ensure that the Conversational Analytics API service agent has the necessary permissions to create the log table and write conversation logs to it, ask your administrator to grant the BigQuery Data Editor (roles/bigquery.dataEditor) IAM role to the Conversational Analytics API service agent on the log dataset.

For more information about granting roles, see Manage access to projects, folders, and organizations.

Your administrator might also be able to give the Conversational Analytics API service agent the required permissions through custom roles or other predefined roles.

Enable conversation logging

Conversation logging needs a dataset that the service agent can write to. Create the dataset and grant access first, and then turn on logging for the data agent.

Create the log dataset

Create a dataset in the data agent's project to hold the logs. The service agent creates the log table in this dataset on the first logged turn. To create the dataset, use the bq mk --dataset command in the bq command-line tool. You can run bq tool commands in Cloud Shell or in a local terminal where the Google Cloud CLI is installed.

bq --location=LOCATION mk --dataset PROJECT_ID:LOG_DATASET

Replace the sample values as follows:

  • LOCATION: The BigQuery location for the log dataset, such as US.
  • PROJECT_ID: The ID of the project that contains the data agent.
  • LOG_DATASET: The ID of the dataset to create.

For other ways to create a dataset, see Create datasets.

Grant the service agent access to the dataset

Grant the service agent the BigQuery Data Editor role (roles/bigquery.dataEditor) on the log dataset only.

Select one of the following options:

Console

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, expand your project, click Datasets, and then select the log dataset.

  3. Click Share > Manage permissions.

  4. Click Add principal.

  5. In the New principals field, enter the service agent's email address: service-PROJECT_NUMBER@gcp-sa-geminidataanalytics.iam.gserviceaccount.com.

    Replace PROJECT_NUMBER with the number of the project that contains the data agent.

  6. In the Select a role list, select BigQuery Data Editor.

  7. Click Save.

SQL

In the BigQuery query editor, set the query location to the log dataset's location, and then run the following GRANT statement:

GRANT `roles/bigquery.dataEditor`
ON SCHEMA `PROJECT_ID`.LOG_DATASET
TO "serviceAccount:service-PROJECT_NUMBER@gcp-sa-geminidataanalytics.iam.gserviceaccount.com";

Replace the sample values as follows:

  • PROJECT_ID: The ID of the project that contains the data agent and the log dataset.
  • LOG_DATASET: The ID of the log dataset.
  • PROJECT_NUMBER: The number of the project that contains the data agent.

For other ways to grant access, see Grant access to a dataset. If the grant fails because the service agent doesn't exist, see Granting dataset access fails because the service agent doesn't exist.

Turn on logging for the data agent

To turn on logging for an existing data agent, set the agent's bigquery_agent_analytics_enabled and bigquery_agent_analytics_table fields (bigqueryAgentAnalyticsEnabled and bigqueryAgentAnalyticsTable in HTTP requests), and include both fields in the update mask. The update changes only the fields in the update mask. The service agent writes log rows only when you turn on logging and set a log table.

Run the following samples in the Python environment that you set up in Configure settings and initialize clients (Python SDK) or Configure initial settings and authentication (HTTP), such as a Colaboratory notebook. The Python SDK samples on this page require google-cloud-geminidataanalytics version 0.13.4 or later.

Python SDK

from google.protobuf import field_mask_pb2

data_agent_id = "DATA_AGENT_ID"
log_dataset = "LOG_DATASET"
log_table = "LOG_TABLE"

data_agent = geminidataanalytics.DataAgent()
data_agent.name = f"projects/{billing_project}/locations/{location}/dataAgents/{data_agent_id}"
data_agent.bigquery_agent_analytics_enabled = True
data_agent.bigquery_agent_analytics_table = geminidataanalytics.BigQueryTableReference(
    project_id=billing_project,
    dataset_id=log_dataset,
    table_id=log_table,
)

update_mask = field_mask_pb2.FieldMask(
    paths=["bigquery_agent_analytics_enabled", "bigquery_agent_analytics_table"]
)

request = geminidataanalytics.UpdateDataAgentRequest(
    data_agent=data_agent,
    update_mask=update_mask,
)

try:
  response = data_agent_client.update_data_agent_sync(request=request)
  print(response.bigquery_agent_analytics_enabled)
  print(response.bigquery_agent_analytics_table)
except Exception as e:
  print(f"Error updating data agent: {e}")

Replace the sample values as follows:

  • DATA_AGENT_ID: The ID of the data agent.
  • LOG_DATASET: The ID of the log dataset.
  • LOG_TABLE: The ID of the log table, such as agent_events. If the table doesn't exist, the service agent creates it on the first logged turn.

HTTP

data_agent_id = "DATA_AGENT_ID"
log_dataset = "LOG_DATASET"
log_table = "LOG_TABLE"

data_agent_url = f"{base_url}/v1/projects/{billing_project}/locations/{location}/dataAgents/{data_agent_id}:updateSync"

payload = {
    "bigqueryAgentAnalyticsEnabled": True,
    "bigqueryAgentAnalyticsTable": {
        "projectId": billing_project,
        "datasetId": log_dataset,
        "tableId": log_table,
    },
}

params = {
    "updateMask": "bigqueryAgentAnalyticsEnabled,bigqueryAgentAnalyticsTable"
}

data_agent_response = requests.patch(
    data_agent_url, headers=headers, params=params, json=payload
)

if data_agent_response.status_code == 200:
    body = data_agent_response.json()
    if body.get("bigqueryAgentAnalyticsEnabled") and body.get(
        "bigqueryAgentAnalyticsTable"
    ):
        print("Conversation logging is turned on.")
    else:
        print("Logging isn't turned on. Check the returned data agent.")
    print(json.dumps(body, indent=2))
else:
    print(f"Error updating data agent: {data_agent_response.status_code}")
    print(data_agent_response.text)

Replace the sample values as follows:

  • DATA_AGENT_ID: The ID of the data agent.
  • LOG_DATASET: The ID of the log dataset.
  • LOG_TABLE: The ID of the log table, such as agent_events. If the table doesn't exist, the service agent creates it on the first logged turn.

The method returns the updated data agent. Confirm that logging is enabled and that the log table is set.

To turn on logging when you create a data agent, include the same fields in your create request. For the create request, see Create a data agent (Python SDK or HTTP). In an HTTP request body, the fields look like the following:

"bigqueryAgentAnalyticsEnabled": true,
"bigqueryAgentAnalyticsTable": {
  "projectId": "PROJECT_ID",
  "datasetId": "LOG_DATASET",
  "tableId": "LOG_TABLE"
}

Replace the sample values as follows:

  • PROJECT_ID: The ID of the project that contains the data agent.
  • LOG_DATASET: The ID of the log dataset.
  • LOG_TABLE: The ID of the log table.

Verify that conversations are logged

  1. Send a chat request to the data agent. For examples, see Use the API to ask questions (Python SDK or HTTP).
  2. Wait about 10 seconds after the agent answers. Then, in Cloud Shell or a terminal, run the following bq query command to see recent questions and answers. You can also run the SQL in the BigQuery query editor.

    bq query --use_legacy_sql=false '
    SELECT timestamp, event_type, JSON_VALUE(content, "$.text_summary") AS text
    FROM `PROJECT_ID.LOG_DATASET.LOG_TABLE`
    WHERE timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 HOUR)
      AND event_type IN ("USER_MESSAGE_RECEIVED", "AGENT_RESPONSE")
    ORDER BY timestamp DESC
    LIMIT 20'
    

    Replace the sample values as follows:

    • PROJECT_ID: The ID of the project that contains the data agent.
    • LOG_DATASET: The ID of the log dataset.
    • LOG_TABLE: The ID of the log table.

    If the query returns your question and the agent's answer, conversation logging is working correctly. If the table doesn't exist or is empty, see Troubleshoot conversation logging.

Change the destination table

To write logs to a different table, update the data agent's bigquery_agent_analytics_table field (bigqueryAgentAnalyticsTable in HTTP requests), and include the field in the update mask. The new table must be in the data agent's project, and the service agent must have the BigQuery Data Editor role on the new table's dataset.

Run the following sample in the same environment as in Turn on logging for the data agent.

Python SDK

from google.protobuf import field_mask_pb2

data_agent_id = "DATA_AGENT_ID"
log_dataset = "NEW_LOG_DATASET"
log_table = "NEW_LOG_TABLE"

data_agent = geminidataanalytics.DataAgent()
data_agent.name = f"projects/{billing_project}/locations/{location}/dataAgents/{data_agent_id}"
data_agent.bigquery_agent_analytics_table = geminidataanalytics.BigQueryTableReference(
    project_id=billing_project,
    dataset_id=log_dataset,
    table_id=log_table,
)

update_mask = field_mask_pb2.FieldMask(paths=["bigquery_agent_analytics_table"])

request = geminidataanalytics.UpdateDataAgentRequest(
    data_agent=data_agent,
    update_mask=update_mask,
)

try:
  response = data_agent_client.update_data_agent_sync(request=request)
  print(response.bigquery_agent_analytics_table)
except Exception as e:
  print(f"Error updating data agent: {e}")

Replace the sample values as follows:

  • DATA_AGENT_ID: The ID of the data agent.
  • NEW_LOG_DATASET: The ID of the new log dataset.
  • NEW_LOG_TABLE: The ID of the new log table.

HTTP

data_agent_id = "DATA_AGENT_ID"
log_dataset = "NEW_LOG_DATASET"
log_table = "NEW_LOG_TABLE"

data_agent_url = f"{base_url}/v1/projects/{billing_project}/locations/{location}/dataAgents/{data_agent_id}:updateSync"

payload = {
    "bigqueryAgentAnalyticsTable": {
        "projectId": billing_project,
        "datasetId": log_dataset,
        "tableId": log_table,
    },
}

params = {"updateMask": "bigqueryAgentAnalyticsTable"}

data_agent_response = requests.patch(
    data_agent_url, headers=headers, params=params, json=payload
)

if data_agent_response.status_code == 200:
    body = data_agent_response.json()
    table = body.get("bigqueryAgentAnalyticsTable", {})
    if table.get("datasetId") == log_dataset and table.get("tableId") == log_table:
        print("Log table updated.")
    else:
        print("The log table didn't change. Check the returned data agent.")
    print(json.dumps(body, indent=2))
else:
    print(f"Error updating data agent: {data_agent_response.status_code}")
    print(data_agent_response.text)

Replace the sample values as follows:

  • DATA_AGENT_ID: The ID of the data agent.
  • NEW_LOG_DATASET: The ID of the new log dataset.
  • NEW_LOG_TABLE: The ID of the new log table.

Rows that were already logged stay in the previous table. The change takes effect on the next turn.

Turn off conversation logging

To turn off conversation logging for a data agent, set the agent's bigquery_agent_analytics_enabled field (bigqueryAgentAnalyticsEnabled in HTTP requests) to false, and include the field in the update mask.

Run the following sample in the same environment as in Turn on logging for the data agent.

Python SDK

from google.protobuf import field_mask_pb2

data_agent_id = "DATA_AGENT_ID"

data_agent = geminidataanalytics.DataAgent()
data_agent.name = f"projects/{billing_project}/locations/{location}/dataAgents/{data_agent_id}"
data_agent.bigquery_agent_analytics_enabled = False

update_mask = field_mask_pb2.FieldMask(paths=["bigquery_agent_analytics_enabled"])

request = geminidataanalytics.UpdateDataAgentRequest(
    data_agent=data_agent,
    update_mask=update_mask,
)

try:
  response = data_agent_client.update_data_agent_sync(request=request)
  print(response.bigquery_agent_analytics_enabled)
except Exception as e:
  print(f"Error updating data agent: {e}")

Replace DATA_AGENT_ID with the ID of the data agent.

HTTP

data_agent_id = "DATA_AGENT_ID"

data_agent_url = f"{base_url}/v1/projects/{billing_project}/locations/{location}/dataAgents/{data_agent_id}:updateSync"

payload = {"bigqueryAgentAnalyticsEnabled": False}
params = {"updateMask": "bigqueryAgentAnalyticsEnabled"}

data_agent_response = requests.patch(
    data_agent_url, headers=headers, params=params, json=payload
)

if data_agent_response.status_code == 200:
    print("Conversation logging is turned off.")
    print(json.dumps(data_agent_response.json(), indent=2))
else:
    print(f"Error updating data agent: {data_agent_response.status_code}")
    print(data_agent_response.text)

Replace DATA_AGENT_ID with the ID of the data agent.

The change takes effect on the next turn. A turn that's already running finishes with the settings that were in place when it started.

To stop logging for all data agents that write to a log dataset, revoke the service agent's BigQuery Data Editor role on the dataset. Logging stays turned on for each agent, but the service agent can no longer write rows.

Turning off logging keeps the rows that were already logged. To remove logged data, delete rows, partitions, or the log table.

Troubleshoot conversation logging

Use the following information to troubleshoot common issues with conversational analytics conversation logging in BigQuery. Because logging errors never fail a conversation turn and aren't reported in the chat, in the API response, or in Cloud Logging, an empty or missing destination table is the primary signal of a configuration or permission issue.

Log table is missing or empty

If the destination table isn't created or remains empty after a conversation turn, check the following potential causes:

  • Conversation logging isn't turned on. Confirm that conversation logging is enabled and the destination table is configured on the data agent.

  • Destination dataset is missing or in a different project. Confirm that the destination dataset already exists in the same project as the data agent. You can verify that the dataset exists by running bq show PROJECT_ID:LOG_DATASET. Replace PROJECT_ID with the ID of the project that contains the data agent, and LOG_DATASET with the ID of the destination dataset. The service agent creates the log table on the first logged turn, but it doesn't create the dataset.

  • Destination dataset is a read-only dataset, such as a linked dataset. A BigQuery sharing linked dataset or an external dataset (for example, an AWS Glue federated dataset) is read-only and can't store conversation logs. To check the dataset type, run bq show --format=prettyjson PROJECT_ID:LOG_DATASET and inspect the "type" field ("LINKED" or "EXTERNAL"). Choose a standard writable dataset in the same project as the data agent.

  • Missing service agent or CMEK permissions. Confirm that the Conversational Analytics API service agent (service-PROJECT_NUMBER@gcp-sa-geminidataanalytics.iam.gserviceaccount.com, using the agent project's number rather than the project ID) has the BigQuery Data Editor (roles/bigquery.dataEditor) role on the log dataset. If the dataset uses customer-managed Cloud KMS keys (CMEK), confirm that the BigQuery encryption service account (bq-PROJECT_NUMBER@bigquery-encryption.iam.gserviceaccount.com) has the roles/cloudkms.cryptoKeyEncrypterDecrypter role on the key.

  • VPC Service Controls perimeter blocks writes. The Conversational Analytics API service agent writes to the log table from Google-managed infrastructure, so granting roles/bigquery.dataEditor alone isn't enough when a service perimeter restricts bigquery.googleapis.com. Unlike other causes on this list, a perimeter block records Policy Denied audit logs in Cloud Logging in the agent's project; filter by log_id("cloudaudit.googleapis.com/policy") under bigquery.googleapis.com (for table lookup or creation) and bigquerystorage.googleapis.com (for row appends) to confirm the block. Confirm that both geminidataanalytics.googleapis.com and bigquery.googleapis.com are in the perimeter, and add an ingress rule for the service agent on bigquery.googleapis.com. For a sample ingress rule, see Access control, encryption, and network security. For general perimeter setup guidance, see Configure perimeter security with VPC Service Controls.

  • Unsupported conversation surface or agent configuration. Conversations in the BigQuery Studio Preview pane and stateless API calls without a data agent aren't logged. To test, send a chat request to the data agent through the API. Agents that use a search scope instead of specific BigQuery tables, that configure only BigQuery property graphs (without BigQuery tables) as knowledge sources, or that don't declare any BigQuery tables, aren't logged. For more information, see Limitations.

  • Knowledge-source tables use access policies. If any knowledge-source table (a BigQuery table in the agent's data sources) uses row-level access policies, column-level security (policy tags), or dynamic data masking, turns aren't logged.

  • Schema conflict on an existing destination table. If you configured an existing table as the destination and a column shares a name with a conversation log schema column but has a different type or mode, BigQuery doesn't write rows to that table.

Rows typically appear within a few seconds after a turn completes. If rows still don't appear after you wait a few minutes, refresh the dataset.

Some users' conversations aren't logged

Cause

Before each turn, the privacy check uses the chatting user's credentials to verify that the agent's knowledge-source tables don't have row-level security, column-level security, or data masking policies. If a user lacks permission to run this check, their turns aren't logged.

Resolution

Grant all chatting users the BigQuery Data Viewer (roles/bigquery.dataViewer) role on the knowledge-source tables and the BigQuery Job User (roles/bigquery.jobUser) role on the project that runs their queries, or grant a custom role with the bigquery.tables.get and bigquery.rowAccessPolicies.list permissions on the knowledge-source tables.

Creating or updating the agent fails with an INVALID_ARGUMENT error

Error message

  • The BigQuery Agent Analytics table must be in the same project as the agent (agent project: ..., table project: ...).

  • BigQuery Agent Analytics can only be enabled on agents whose datasource is BigQuery. Remove bigquery_agent_analytics_enabled and bigquery_agent_analytics_table, or point the agent at a BigQuery datasource.

Cause

The destination dataset is in a different project from the data agent, or the agent's data source isn't BigQuery.

Resolution

Specify an existing destination dataset in the same project as the data agent, or configure the data agent with a BigQuery data source.

Granting dataset access fails because the service agent doesn't exist

Error message

  • Service account service-PROJECT_NUMBER@gcp-sa-geminidataanalytics... does not exist

Cause

The Conversational Analytics API service agent hasn't been provisioned in the project yet.

Resolution

Do one of the following:

  • Configure the log destination on the agent first, and then grant the role again.

  • Provision the service identity by running the following command:

    gcloud beta services identity create \
        --service=geminidataanalytics.googleapis.com \
        --project=PROJECT_ID

    Replace PROJECT_ID with the ID of the project that contains the data agent.

What's next