Data management controls for generative AI use cases

This document includes the best practices and guidelines for data management systems when running generative AI workloads on Google Cloud.

Data management systems such as BigQuery and Cloud Storage let you store the data that you require for your ML workflows, including training data, model artifacts, and production data.

Common controls

These controls apply to all data management systems.

Enable Sensitive Data Protection for data inspection

Google control ID COM-CO-5.1
Implementation Recommended
Description

Google Cloud recommends using Sensitive Data Protection. The infoTypes or job templates that you select depend on your particular systems.
Applicable products
  • Sensitive Data Protection
Related NIST-800-53 controls
  • SI-4
  • IA-7
  • SC-7
  • SC-8
Related CRI profile controls
  • PR.AC-1.1
  • PR.AC-1.2
  • PR.AC-1.3
  • PR.DS-5.1
  • PR.DS-8.1
  • ID.RA-1.1
  • DE.CM-1.1
  • DE.CM-1.2
  • DE.CM-1.3
  • DE.CM-1.4
  • DE.CM-5.1
  • DE.CM-6.1
  • DE.CM-6.2
  • DE.CM-6.3
  • DE.CM-7.1
  • DE.CM-7.2
  • DE.CM-7.3
  • DE.CM-7.4
  • DE.DP-2.1
  • DE.DP-3.1
  • DE.DP-4.1
  • DE.DP-4.2
  • DE.DP-5.1
  • DE.AE-2.1
  • DE.AE-3.1
  • DE.AE-3.2
  • DE.AE-4.1
Related information

Data warehouse controls

These controls apply to BigQuery.

Ensure BigQuery datasets aren't publicly readable or set to allAuthenticatedUsers

Google control ID BQ-CO-6.1
Implementation Required
Description

Restrict access to the information in a BigQuery dataset to specific users only. To configure this protection, you must set up detailed roles.
Applicable products
  • Organization Policy Service
  • BigQuery
  • Identity and Access Management (IAM)
Path cloudasset.assets/assetType
Operator ==
Value
  • bigquery.googleapis.com/Dataset
Type String
Related NIST-800-53 controls
  • AC-3
  • AC-12
  • AC-17
  • AC-20
Related CRI profile controls
  • PR.AC-3.1
  • PR.AC-3.2
  • PR.AC-4.1
  • PR.AC-4.2
  • PR.AC-4.3
  • PR.AC-6.1
  • PR.AC-7.1
  • PR.AC-7.2
  • PR.PT-3.1
  • PR-PT-4.1
Related information

Ensure BigQuery tables aren't publicly readable or set to allAuthenticatedUsers

Google control ID BQ-CO-6.2
Implementation Required
Description

Restrict access to the information in a BigQuery table to specific users only. To configure this protection, you must set up detailed roles.
Applicable products
  • Identity and Access Management (IAM)
  • BigQuery
Path cloudasset.assets/iamPolicy.bindings.members
Operator anyof
Value
  • allUsers
  • allAuthenticatedUsers
Type String
Related NIST-800-53 controls
  • AC-3
  • AC-12
  • AC-17
  • AC-20
Related CRI profile controls
  • PR.AC-3.1
  • PR.AC-3.2
  • PR.AC-4.1
  • PR.AC-4.2
  • PR.AC-4.3
  • PR.AC-6.1
  • PR.AC-7.1
  • PR.AC-7.2
  • PR.PT-3.1
  • PR-PT-4.1
Related information

Encrypt individual values in a BigQuery table

Google control ID BQ-CO-6.3
Implementation Optional
Description

If your organization requires that you encrypt individual values within a BigQuery table, use the Authenticated Encryption with Associated Data (AEAD) encryption functions.
Applicable products
  • BigQuery
Related NIST-800-53 controls
  • SC-13
Related CRI profile controls
  • PR.DS-5.1
Related information

Use authorized views for BigQuery datasets

Google control ID BQ-CO-6.4
Implementation Optional
Description

Authorized views let you share a subset of data in a dataset to specific users. For example, an authorized view lets you share query results with particular users and groups without giving them access to the underlying source data.
Applicable products
  • BigQuery
Related NIST-800-53 controls
  • AC-3
  • AC-12
  • AC-17
  • AC-20
Related CRI profile controls
  • PR.AC-3.1
  • PR.AC-3.2
  • PR.AC-4.1
  • PR.AC-4.2
  • PR.AC-4.3
  • PR.AC-6.1
  • PR.AC-7.1
  • PR.AC-7.2
  • PR.PT-3.1
  • PR-PT-4.1
Related information

Use BigQuery column-level security

Google control ID BQ-CO-6.5
Implementation Optional
Description

Use BigQuery column-level security to create policies that check at query time whether a user has proper access. BigQuery provides fine-grained access to sensitive columns using policy tags or type-based classification of data.
Applicable products
  • BigQuery
Related NIST-800-53 controls
  • AC-3
  • AC-12
  • AC-17
  • AC-20
Related CRI profile controls
  • PR.AC-3.1
  • PR.AC-3.2
  • PR.AC-4.1
  • PR.AC-4.2
  • PR.AC-4.3
  • PR.AC-6.1
  • PR.AC-7.1
  • PR.AC-7.2
  • PR.PT-3.1
  • PR-PT-4.1
Related information

Use BigQuery row-level security

Google control ID BQ-CO-6.6
Implementation Optional
Description

Use row-level security and access policies to enable fine-grained access control to a subset of data in a BigQuery table.
Applicable products
  • BigQuery
Related NIST-800-53 controls
  • AC-3
  • AC-12
  • AC-17
  • AC-20
Related CRI profile controls
  • PR.AC-3.1
  • PR.AC-3.2
  • PR.AC-4.1
  • PR.AC-4.2
  • PR.AC-4.3
  • PR.AC-6.1
  • PR.AC-7.1
  • PR.AC-7.2
  • PR.PT-3.1
  • PR-PT-4.1
Related information

Use BigQuery resource charts

Google control ID BQ-CO-7.1
Implementation Optional
Description

BigQuery resource charts let BigQuery administrators observe how their organization, folder, or reservation uses BigQuery slots and how their queries perform.
Applicable products
  • BigQuery
Related NIST-800-53 controls
  • AC-3
  • AC-12
  • AC-17
  • AC-20
Related CRI profile controls
  • PR.AC-3.1
  • PR.AC-3.2
  • PR.AC-4.1
  • PR.AC-4.2
  • PR.AC-4.3
  • PR.AC-6.1
  • PR.AC-7.1
  • PR.AC-7.2
  • PR.PT-3.1
  • PR-PT-4.1
Related information

Storage controls

These controls apply to Cloud Storage.

Block public access to Cloud Storage buckets

Google control ID GCS-CO-4.1
Implementation Required
Description

The storage.publicAccessPrevention boolean constraint prevents storage buckets from being accessed from public sources without authentication. It disables and blocks access control lists (ACLs) and Identity and Access Management (IAM) permissions that grant access to allUsers and allAuthenticatedUsers. This constraint acts as an organization-wide safety net that actively blocks any setting that would make a bucket publicly accessible.
Applicable products
  • Organization Policy Service
  • Cloud Storage
Path constraints/storage.publicAccessPrevention
Operator ==
Value
  • True
Type Boolean
Related NIST-800-53 controls
  • AC-3
  • AC-17
  • AC-20
Related CRI profile controls
  • PR.AC-3.1
  • PR.AC-3.2
  • PR.AC-4.1
  • PR.AC-4.2
  • PR.AC-4.3
  • PR.AC-6.1
  • PR.PT-3.1
  • PR.PT-4.1
Related information

Use uniform bucket-level access

Google control ID GCS-CO-4.2
Implementation Required
Description

The storage.uniformBucketLevelAccess boolean constraint requires buckets to use uniform bucket-level access. Uniform bucket-level access lets you only use bucket-level Identity and Access Management (IAM) permissions to grant access to your Cloud Storage resources.

Using two different and conflicting systems to manage permissions on storage buckets is complex and a common cause of accidental data leaks. This setting turns off the legacy system (access control lists, or ACLs) and makes the modern, centralized system (IAM) the single source of truth for all permissions.
Applicable products
  • Organization Policy Service
  • Cloud Storage
Path constraints/storage.uniformBucketLevelAccess
Operator ==
Value
  • True
Type Boolean
Related NIST-800-53 controls
  • AC-3
  • AC-17
  • AC-20
Related CRI profile controls
  • PR.AC-3.1
  • PR.AC-3.2
  • PR.AC-4.1
  • PR.AC-4.2
  • PR.AC-4.3
  • PR.AC-6.1
  • PR.PT-3.1
  • PR.PT-4.1
Related information

Protect HMAC keys for service accounts

Google control ID GCS-CO-6.9
Implementation Required
Description

An HMAC key is a long-lived type of credential that is associated with a service account or a user account in Cloud Storage. Use an HMAC key to create signatures that are included in requests to Cloud Storage. A signature proves a user or service account has authorized a request.

Unlike short-lived credentials (such as. OAuth 2.0 tokens), HMAC keys don't expire automatically and remain valid until manually revoked. HMAC keys are high-risk credentials: if compromised, they provide persistent access to your resources. You must ensure appropriate mechanisms are in place to help protect them.
Applicable products
  • Cloud Storage
Path storage.projects.hmacKeys/id
Operator Exists
Value
  • []
Type String
Related NIST-800-53 controls
  • SC-12
  • SC-13
Related CRI profile controls
  • PR.DS-1.1
  • PR.DS-1.2
  • PR.DS-2.1
  • PR.DS-2.2
  • PR.DS-5.1
Related information

Detect enumeration of Cloud Storage buckets by service accounts

Google control ID GCS-CO-7.2
Implementation Required
Description

Service accounts are non-human identities that are designed for applications, and their behavior is predictable and automated. Normally, service accounts don't need to itemize buckets, as they're already mapped. Therefore, if you detect a service account attempting to retrieve a list of all Cloud Storage buckets, investigate it immediately. Reconnaissance enumeration is often used as a recon technique by a malicious actor that has gained access to the service account.
Applicable products
  • Cloud Storage
  • Cloud Audit Logs
Operator ==
Value
  • storage.bucket.list
Type String
Related NIST-800-53 controls
  • AU-2
  • AU-3
  • AU-8
  • AU-9
Related CRI profile controls
  • DM.ED-7.1
  • DM.ED-7.2
  • DM.ED-7.3
  • DM.ED-7.4
  • PR.IP-1.4
Related information

Detect Identity and Access Management (IAM) policy modifications of Cloud Storage buckets by service accounts

Google control ID GCS-CO-7.3
Implementation Required
Description

Configure an alert that detects when a Cloud Storage bucket's IAM policy is modified to grant public access. This alert fires when the allUsers or allAuthenticatedUsers principals are added to a bucket's IAM policy. This alert is a critical, high-severity event because it can expose all data in the bucket. Investigate this alert immediately to confirm if the change was authorized or is a sign of a misconfiguration or malicious actor.

In the alert, set the data.protoPayload.serviceData.policyData.bindingDeltas.member JSON attribute to allUsers or allAuthenticatedUsers and the action to ADD.
Applicable products
  • Cloud Storage
  • Cloud Audit Logs
Related NIST-800-53 controls
  • AU-2
  • AU-3
  • AU-8
  • AU-9
Related CRI profile controls
  • DM.ED-7.1
  • DM.ED-7.2
  • DM.ED-7.3
  • DM.ED-7.4
  • PR.IP-1.4
Related information

Ensure Cloud Storage bucket retention policy uses Bucket Lock

Google control ID GCS-CO-6.1
Implementation Recommended
Description

Depending on your regulatory requirements, ensure that each Cloud Storage bucket retention policy is locked. Set the retention period to a timeframe that meets your requirements.
Applicable products
  • Cloud Storage
Path storage.buckets/retentionPolicy.isLocked
Operator !=
Value
  • True
Type Boolean
Related NIST-800-53 controls
  • SI-12
Related CRI profile controls
  • PR.IP-2.1
  • PR.IP-2.2
  • PR.IP-2.3
Related information

Set lifecycle rules for the SetStorageClass action

Google control ID GCS-CO-6.11
Implementation Recommended
Description

Apply lifecycle rules to each Cloud Storage bucket that has a SetStorageClass action type.
Applicable products
  • Cloud Storage
Path storage.buckets/lifecycle.rule.action.type
Operator ==
Value
  • SetStorageClass
Type String
Related NIST-800-53 controls
  • SI-12
Related CRI profile controls
  • PR.IP-2.1
  • PR.IP-2.2
  • PR.IP-2.3
Related information

Set permitted regions for storage classes

Google control ID GCS-CO-6.12
Implementation Recommended
Description
Ensure that storage classes for the lifecycle configuration aren't within permitted regional classifications.
Applicable products
  • Cloud Storage
Path storage.buckets/lifecycle.rule.action.storageClass
Operator nin
Value
  • MULTI_REGIONAL
  • REGIONAL
Type String
Related NIST-800-53 controls
  • SI-12
Related CRI profile controls
  • PR.IP-2.1
  • PR.IP-2.2
  • PR.IP-2.3
Related information

Enable lifecycle management for Cloud Storage buckets

Google control ID GCS-CO-6.13
Implementation Recommended
Description

Ensure that lifecycle management of Cloud Storage is enabled and configured. The lifecycle control contains the configuration for the storage lifecycle. Verify that the policies in this setting match your requirements.
Applicable products
  • Cloud Storage
Path storage.buckets/lifecycle
Operator Exists
Value
  • []
Type Object
Related NIST-800-53 controls
  • SI-12
Related CRI profile controls
  • PR.IP-2.1
  • PR.IP-2.2
  • PR.IP-2.3
Related information

Enable lifecycle management rules for Cloud Storage buckets

Google control ID GCS-CO-6.14
Implementation Recommended
Description

Ensure that lifecycle management rules for Cloud Storage are enabled and configured. The rule control contains the configuration for the storage lifecycle. Verify that the policies in this setting match your requirements.
Applicable products
  • Cloud Storage
Path storage.buckets/lifecycle.rule
Operator Empty
Value
  • []
Type Array
Related NIST-800-53 controls
  • SI-12
Related CRI profile controls
  • PR.IP-2.1
  • PR.IP-2.2
  • PR.IP-2.3
Related information

Review and evaluate temporary holds on active objects

Google control ID GCS-CO-6.16
Implementation Recommended
Description

Identify all objects where temporaryHold is set to TRUE and start an investigation and validation process. This evaluation is appropriate for the following use cases:

  • Legal hold: To comply with legal requirements for storing data, temporary hold can be used to prevent the deletion of sensitive data that may be relevant to ongoing investigations or litigation.
  • Data loss prevention: To prevent accidental deletion of important data, temporary hold can be used as a safety measure to protect critical business information.
  • Content moderation: To review potentially sensitive or inappropriate content before it becomes publicly accessible, apply a temporary hold to content uploaded to Cloud Storage for further inspection and moderation decisions.
Applicable products
  • Cloud Storage
Path storage.objects/temporaryHold
Operator ==
Value
  • TRUE
Type Boolean
Related NIST-800-53 controls
  • SI-12
Related CRI profile controls
  • PR.IP-2.1
  • PR.IP-2.2
  • PR.IP-2.3
Related information

Enforce retention policies on Cloud Storage buckets

Google control ID GCS-CO-6.17
Implementation Recommended
Description

Ensure that all the Cloud Storage buckets have a retention policy.
Applicable products
  • Cloud Storage
Path storage.buckets/retentionPolicy.retentionPeriod
Operator agesmaller
Value
  • [90,"DAY","AFTER","yyyy-MM-dd'T'HH:mm:ss'Z'"]
Type int64
Related NIST-800-53 controls
  • SI-12
Related CRI profile controls
  • PR.IP-2.1
  • PR.IP-2.2
  • PR.IP-2.3
Related information

Enforce classification tags for Cloud Storage buckets

Google control ID GCS-CO-6.18
Implementation Recommended
Description

Data classification is a foundational component of any data governance and security program. Applying a classification label with values like public, internal, confidential, or restricted to each bucket is essential.

Confirm that google_storage_bucket.labels has an expression for classification and create a violation if it doesn't.
Applicable products
  • Cloud Storage
Path storage.buckets/labels.classification
Operator notexists
Value
  • []
Type Extended
Related NIST-800-53 controls
  • SI-12
Related CRI profile controls
  • PR.IP-2.1
  • PR.IP-2.2
  • PR.IP-2.3
Related information

Enforce log buckets for Cloud Storage buckets

Google control ID GCS-CO-6.3
Implementation Recommended
Description

Ensure that every Cloud Storage bucket includes a log bucket.
Applicable products
  • Cloud Storage
Path storage.buckets/logging.logBucket
Operator notexists
Value
  • []
Type String
Related NIST-800-53 controls
  • AU-2
  • AU-3
  • AU-8
  • AU-9
Related CRI profile controls
  • DM.ED-7.1
  • DM.ED-7.2
  • DM.ED-7.3
  • DM.ED-7.4
  • PR.IP-1.4
Related information

Configure deletion rules for Cloud Storage buckets

Google control ID GCS-CO-6.5
Implementation Recommended
Description

In Cloud Storage, storage.buckets/lifecycle.rule.action.type refers to the type of action to be taken on a specific object based on a lifecycle rule within a bucket. This configuration helps automate the management and lifecycle of your data stored in the cloud.

Configure the storage.buckets/lifecycle.rule.action.type to ensure that objects are permanently deleted from the bucket.
Applicable products
  • Cloud Storage
Path storage.buckets/lifecycle.rule.action.type
Operator ==
Value
  • Delete
Type String
Related NIST-800-53 controls
  • SI-12
Related CRI profile controls
  • PR.IP-2.1
  • PR.IP-2.2
  • PR.IP-2.3
Related information

Ensure isLive condition is False for deletion rules

Google control ID GCS-CO-6.6
Implementation Recommended
Description

For deletion rules, ensure that the isLive condition of the rule is set to false.

In Cloud Storage, storage.buckets/lifecycle.rule.condition.isLive is a boolean condition that is used in lifecycle rules to determine whether an object is considered live. This filter helps ensure that actions within a lifecycle rule are applied only to desired objects based on their live status.

Use cases:

  • Archive historical versions: Archive only non-current versions of objects to save storage costs while keeping the latest version readily accessible.
  • Clean up deleted objects: Automate permanent deletion of objects that have been deleted by users, freeing up space in the bucket.
  • Protect live data: Ensure that actions like setting temporary holds are applied only to live objects, preventing accidental modification of archived or deleted versions
Applicable products
  • Cloud Storage
Path storage.buckets/lifecycle.rule.condition.isLive
Operator ==
Value
  • False
Type Boolean
Related NIST-800-53 controls
  • SI-12
Related CRI profile controls
  • PR.IP-2.1
  • PR.IP-2.2
  • PR.IP-2.3
Related information

Enforce versioning for Cloud Storage buckets

Google control ID GCS-CO-6.7
Implementation Recommended
Description

Ensure that all Cloud Storage buckets have versioning enabled. Use cases include the following:

  • Data protection and recovery: Protect against accidental data loss by preventing overwrites and enabling recovery of deleted or modified data.
  • Compliance and auditing: Maintain a history of all object edits for regulatory compliance or internal auditing purposes.
  • Version control: Track changes to files and data sets, enabling collaboration and rollback to previous versions if necessary.
Applicable products
  • Cloud Storage
Path storage.buckets/versioning.enabled
Operator !=
Value
  • True
Type Boolean
Related NIST-800-53 controls
  • SI-12
Related CRI profile controls
  • PR.IP-2.1
  • PR.IP-2.2
  • PR.IP-2.3
Related information

Enforce owners for Cloud Storage buckets

Google control ID GCS-CO-6.8
Implementation Recommended
Description

Ensure that google_storage_bucket.labels has an expression for an owner.
Applicable products
  • Cloud Storage
Path storage.buckets/labels.owner
Operator notexists
Value
  • []
Type Extended
Related NIST-800-53 controls
  • SI-12
Related CRI profile controls
  • PR.IP-2.1
  • PR.IP-2.2
  • PR.IP-2.3
Related information

Enable logging of key Cloud Storage activities

Google control ID GCS-CO-7.4
Implementation Recommended
Description

Enable additional logging around particular storage objects based on their use case. For example, log access to sensitive data buckets so that you can trace who gained access and when. When enabling additional logging, consider the volume of logs that you might generate.
Applicable products
  • Cloud Storage
Related NIST-800-53 controls
  • AU-2
  • AU-3
  • AU-8
  • AU-9
Related CRI profile controls
  • DM.ED-7.1
  • DM.ED-7.2
  • DM.ED-7.3
  • DM.ED-7.4
  • PR.IP-1.4
Related information

What's next