This document includes the best practices and guidelines for Google Cloud services such as Compute Engine, Google Kubernetes Engine (GKE), Pub/Sub, Dataflow, and Cloud Run functions when running workloads on Google Cloud.
Compute controls
These controls apply to compute services.
Define VM instances that can enable IP forwarding
| Google control ID | VPC-CO-6.3 |
|---|---|
| Implementation | Required |
| Description | The compute.vmCanIpForward constraint defines the VM instances that can enable IP forwarding. By default, any VM can enable IP forwarding in any virtual network. Specify VM instances using one of the following formats:
|
| Applicable products |
|
| Path | constraints/compute.vmCanIpForward |
| Operator | = |
| Value |
|
| Type | List |
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Disable VM-nested virtualization
| Google control ID | VPC-CO-6.6 |
|---|---|
| Implementation | Required |
| Description | The compute.disableNestedVirtualization boolean constraint disables hardware-accelerated nested virtualization for Compute Engine VMs. |
| Applicable products |
|
| Path | constraints/compute.disableNestedVirtualization |
| Operator | Is |
| Value |
|
| Type | Boolean |
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Restrict external IP addresses on VMs
| Google control ID | VPC-CO-6.2 |
|---|---|
| Implementation | Required |
| Description | Unless needed, prevent the creation of Compute Engine instances with public IP addresses. The Prevent Compute Engine instances from having external IP addresses to drastically reduce their exposure to the internet. Any instance with an external IP address is immediately discoverable and becomes a direct target for automated scans, brute-force attacks, and attempts to exploit vulnerabilities. Instead, require instances to use private IP addresses and manage access through controlled, authenticated, and logged pathways like the Identity-Aware Proxy (IAP) tunnel or a bastion host. Adopting this deny-by-default posture is a foundational security best practice that helps minimize your attack surface and enforces a zero-trust approach to your network. This constraint isn't retroactive. |
| Applicable products |
|
| Path | constraints/compute.vmExternalIpAccess |
| Operator | = |
| Value |
|
| Type | List |
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Define permitted external IP addresses for VM instances
| Google control ID | CBD-CO-6.3 |
|---|---|
| Implementation | Required |
| Description | The |
| Applicable products |
|
| Path | compute.vmExternalIpAccess |
| Operator | = |
| Value |
|
| Type | List |
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Require VPC connector for Cloud Run functions
| Google control ID | CF-CO-4.4 |
|---|---|
| Implementation | Required |
| Description | The |
| Applicable products |
|
| Path | constraints/cloudfunctions.requireVPCConnector |
| Operator | = |
| Value |
|
| Type | Boolean |
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Configure message storage policies
| Google control ID | PS-CO-4.1 |
|---|---|
| Implementation | Optional |
| Description | If you publish messages to the global Pub/Sub endpoint, Pub/Sub automatically stores the messages in the nearest Google Cloud region. To control which regions your messages are stored in, configure a message storage policy on your topic.
Use one of the following ways to configure message storage policies for topics:
|
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Turn off external IP addresses for Dataflow jobs
| Google control ID | DF-CO-6.1 |
|---|---|
| Implementation | Optional |
| Description | Turn off external IP addresses for administrative and monitoring tasks that are related to Dataflow jobs. Instead, configure access to your Dataflow worker VMs using SSH. Enable Private Google Access and specify one of the following options in your Dataflow job:
Where:
|
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Use network tags for firewall rules
| Google control ID | DF-CO-6.2 |
|---|---|
| Implementation | Optional |
| Description | Network tags are text attributes that attach to Compute Engine VMs such as Dataflow worker VMs. Network tags let you make VPC network firewall rules and some custom static routes applicable to specific VM instances. Dataflow supports adding network tags to all worker VMs that run a particular Dataflow job. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Container controls
These controls apply to containers within GKE.
Restrict control plane access
| Google control ID | GKE-CO-1.1 |
|---|---|
| Implementation | Required |
| Description | By default, the Google Kubernetes Engine (GKE) cluster control plane and nodes have internet routable addresses that can be accessed from any IP address. Restrict network access to the control plane by using a DNS-based endpoint and by creating private clusters. The control plane is the management center for a Kubernetes cluster, and exposing it to the internet makes it a prime target for attackers. This configuration makes the control plane private and removes it from the internet. Restricting control plane access helps ensure that only trusted devices within your organization's private network can manage the cluster, drastically reducing the risk of an external attack. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Use least-privilege firewall rules
| Google control ID | GKE-CO-1.2 |
|---|---|
| Implementation | Required |
| Description | When you create firewall rules, use the principle of least privilege to provide access only for the required purpose. Ensure that your firewall rules don't conflict with the GKE default firewall rules when possible. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related information |
Use Google Groups for RBAC
| Google control ID | GKE-CO-1.3 |
|---|---|
| Implementation | Required |
| Description | Use Google Groups for role-based access control (RBAC), which also lets you integrate with your existing user account management practices, such as revoking access when someone leaves your organization. Google Groups for RBAC helps provide efficient management of cluster access using Identity and Access Management (IAM) and Google Groups, which is suitable for most organizations that use Google Groups. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Enable Shielded GKE Nodes
| Google control ID | GKE-CO-1.4 |
|---|---|
| Implementation | Required |
| Description | Enable Shielded GKE Nodes for cryptographic verification of the nodes in your cluster. Shielded GKE Nodes provide strong, verifiable node identity and integrity. Enable Shielded GKE Nodes when creating or updating clusters. Where possible, use Shielded GKE Nodes with secure boot to also authenticate the boot components of your node VMs during the boot process. Don't use secure boot if you need third-party unsigned kernel modules. Shielded GKE Nodes is enabled by default when you create a cluster. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related information |
Use Container-Optimized OS with containerd runtime
| Google control ID | GKE-CO-1.5 |
|---|---|
| Implementation | Required |
| Description | Use Container-Optimized OS to implement a hardened and managed container OS. General-purpose operating systems include many extra programs that aren't needed to run containers and therefore create a larger, unnecessary target for attackers. Container-Optimized OS is a minimal, locked-down operating system that significantly reduces this attack surface by including only what is necessary. As a managed OS, Container-Optimized OS also has security patches that are automatically applied by Google, which help ensure critical vulnerabilities are fixed and reduces your operational workload. An image that includes Container-Optimized OS with containerd (cos_containerd) has containerd as the main container runtime directly integrated with Kubernetes. containerd is the core runtime component of Docker and is designed to deliver core container functionality for the Kubernetes Container Runtime Interface (CRI). It is significantly less complex than the full Docker daemon, and therefore has a smaller attack surface. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Use Workload Identity Federation for GKE
| Google control ID | GKE-CO-1.6 |
|---|---|
| Implementation | Required |
| Description | Use Workload Identity Federation for GKE to securely authenticate to Google Cloud APIs from Google Kubernetes Engine (GKE) workloads. Workload Identity Federation for GKE provides a simpler and safer alternative to using service account keys. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Enable GKE Sandbox
| Google control ID | GKE-CO-1.7 |
|---|---|
| Implementation | Required |
| Description | Use GKE Sandbox to provide an extra layer of security to help prevent untrusted code from affecting the host kernel on your Google Kubernetes Engine (GKE) cluster nodes. GKE Sandbox enhances workload isolation for untrusted or sensitive workloads, providing an additional layer of protection against container escape attacks. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Disable the kubelet read-only port
| Google control ID | GKE-CO-1.9 |
|---|---|
| Implementation | Required |
| Description | Disable kubelet read-only port 10255 and use the more secure port 10250 instead. Kubernetes doesn't perform any authentication or authorization checks on this port. The kubelet serves the same endpoints on the more secure, authenticated port 10250. You can only disable the insecure kubelet read-only port in GKE version 1.26.4-gke.500 or later. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related information |
Use namespace and RBAC to restrict access to cluster resources
| Google control ID | GKE-CO-1.10 |
|---|---|
| Implementation | Required |
| Description | Create separate namespaces or clusters for each team and environment to implement least-privilege access to Kubernetes. Assign cost centers and appropriate labels to each namespace for accountability and chargeback. Only give developers the level of access to their namespace that they need to deploy and manage their application, especially in production. Map out the tasks that your users need to undertake against the cluster and define the permissions that they require to do each task. Assign the appropriate Identity and Access Management (IAM) roles for Google Kubernetes Engine (GKE) to groups and users to provide permissions at the project level and use RBAC to grant permissions on a cluster and namespace level. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related information |
Restrict traffic among pods
| Google control ID | GKE-CO-1.11 |
|---|---|
| Implementation | Required |
| Description | By default, all pods in a cluster can communicate with each other. Control pod-to-pod communication as needed for your workloads. Restricting network access to services makes it much more difficult for attackers to move laterally within your cluster, and also offers services some protection against accidental or deliberate denial of service. There are two ways to control traffic:
|
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related information |
Use admission controllers to enforce policies
| Google control ID | GKE-CO-1.12 |
|---|---|
| Implementation | Required |
| Description | Admission controllers are plugins that govern and enforce how the cluster is used. Enable them to use some of the more advanced security features of Kubernetes because they are an important part of the defense in depth approach to hardening your cluster. By default, pods in Kubernetes can operate with capabilities beyond what they require. Use admission controls to restrict the pod's capabilities to only those that are required for that workload. GKE supports numerous controls for restricting your pods to execute with capabilities that are explicitly granted. For example, Policy Controller is available for clusters in fleets. Kubernetes also has the built-in PodSecurity admission controller that lets you enforce the Pod Security Standards in individual clusters. Policy Controller is a feature of GKE that lets you enforce and validate security on GKE clusters at scale by using declarative policies. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related information |
Restrict the ability for workloads to self-modify
| Google control ID | GKE-CO-1.13 |
|---|---|
| Implementation | Required |
| Description | Certain Kubernetes workloads, especially system workloads, have permission to self-modify. For example, some workloads vertically autoscale themselves. While convenient, self-modification can allow an attacker who has already compromised a node to escalate further in the cluster. For example, an attacker could have a workload on the node change itself to run as a more privileged service account that exists in the same namespace. Don't grant workloads the permission to modify themselves by default. When self-modification is necessary, limit permissions by applying Gatekeeper or Policy Controller constraints, such as NoUpdateServiceAccount from the open source Gatekeeper library, which provides several useful security policies. When you deploy policies, allow the controllers that manage the cluster lifecycle to bypass the policies. Controllers must make changes to the cluster, such as applying cluster upgrades. For example, if you deploy the NoUpdateServiceAccount policy on GKE, you must set the following parameters in the constraint: parameters: allowedGroups: - system:masters allowedUsers: - system:addon-manager |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related information |
Monitor your cluster configurations
| Google control ID | GKE-CO-1.14 |
|---|---|
| Implementation | Required |
| Description | Audit your cluster configurations for deviations from your defined settings. Settings covered in these best practices, as well as other common misconfigurations, can be automatically checked using Security Command Center. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related information |
Enforce Binary Authorization
| Google control ID | BIN-CO-1.1 |
|---|---|
| Implementation | Required |
| Description | Use Binary Authorization to make sure trusted images are deployed to Google Kubernetes Engine (GKE) and Cloud Run. Binary Authorization helps ensure that only verified and trusted container images can be deployed in your clusters, strengthening software supply chain security. |
| Applicable products |
|
| Path | constraints/binaryauthorization.requireBinauthz |
| Operator | == |
| Value |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
VMware Engine controls
Limit Admin role assignments for VMware Engine
| Google control ID | GCVE-CO-3.1 |
|---|---|
| Implementation | Required |
| Description | Only grant Admin roles to the service accounts that deploy and configure Google Cloud VMware Engine private clouds and to a limited number of administrators. Typically, the manual addition or deletion of clusters and nodes is an action that doesn't happen frequently and might have a high impact on billing or the availability of the cluster. Make sure to regularly audit who has been assigned the VMware Engine Service Admin role, either directly on the project that's used for VMware Engine or on one of the parent levels of the resource hierarchy. This audit should include other roles, like the basic Editor and Owner roles that include critical permissions related to VMware Engine. You can use services like the IAM roles recommender to help identify overly privileged roles. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Use the VMware Engine Service Viewer role for least privilege
| Google control ID | GCVE-CO-3.3 |
|---|---|
| Implementation | Required |
| Description | Grant the VMware Engine Service Viewer role to users by default, and only if necessary. The VMware Engine Service Viewer role provides read-only access to Google Cloud VMware Engine on Google Cloud and is designed to be sufficient for most tasks. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Use RBAC and least privilege for vCenter Server Appliance roles
| Google control ID | GCVE-CO-3.4 |
|---|---|
| Implementation | Required |
| Description | When granting VMware-level roles in vCenter Server Appliance, use role-based access controls (RBAC) and the principle of least privilege. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Use identity federation for VMware users
| Google control ID | GCVE-CO-3.5 |
|---|---|
| Implementation | Required |
| Description | Maintain a single identity solution where you can provision and manage user accounts. This recommended practice lets you centralize activities such as identity lifecycle management, group management and password management. Avoid creating a fractured identity strategy. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Grant roles to groups instead of individuals for vCenter Server Appliance
| Google control ID | GCVE-CO-3.6 |
|---|---|
| Implementation | Required |
| Description | When granting VMware-level roles in vCenter Server Appliance, grant roles to the groups that you create in your identity provider. Don't create a fractured identity strategy. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Don't assign the Cloud-Owner-Role to user groups in vSphere
| Google control ID | GCVE-CO-3.7 |
|---|---|
| Implementation | Required |
| Description | When creating user groups in vSphere, don't assign the Cloud-Owner-Role. This role grants administrative control over your private cloud's vCenter environment while restricting certain underlying global infrastructure permissions. User groups with permissions higher than Cloud-Owner-Role are automatically reset to Cloud-Owner-Role. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Avoid using default vCenter and NSX-T service accounts
| Google control ID | GCVE-CO-3.8 |
|---|---|
| Implementation | Required |
| Description | Except for the initial configuration process and in emergency procedures, don't use the default vCenter service account (CloudOwner@gve.local) and the default NSX-T service account administrator (admin). These default service accounts include powerful permissions such as Cloud-Owner-Role and Enterprise Admin. Save the credentials for these service accounts in Secret Manager. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Rotate passwords for default vCenter and NSX-T service accounts every 90 days
| Google control ID | GCVE-CO-3.9 |
|---|---|
| Implementation | Required |
| Description | To prevent and mitigate the unauthorized disclosure of the credentials for the default vCenter service account (CloudOwner@gve.local) and the default NSX-T service account administrator (admin), rotate their passwords every 90 days. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Use the NSX Gateway Firewall to segment north-south traffic
| Google control ID | GCVE-CO-1.2 |
|---|---|
| Implementation | Required |
| Description | Control the north-south network traffic entering and exiting your private clouds using the NSX Gateway Firewall. NSX Gateway Firewalls are north-south firewalls that help protect the perimeters. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Use the NSX Distributed Firewall to segment east-west traffic
| Google control ID | GCVE-CO-1.3 |
|---|---|
| Implementation | Required |
| Description | Control the east-west network traffic within your private cloud using the NSX Distributed Firewall (DFW). By default, all subnets can communicate with each other. Use DFW to define permitted traffic between applications and services inside your application. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Create separate subnets for workloads with different security requirements
| Google control ID | GCVE-CO-1.4 |
|---|---|
| Implementation | Required |
| Description | If you run workloads that have different security requirements or data classifications, create workload subnets to separate them. Subnets let you reduce the potential blast radius by not mixing workloads of varying security requirements and postures. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Create a log sink to store VMware Engine audit logs
| Google control ID | GCVE-CO-4.1 |
|---|---|
| Implementation | Required |
| Description | Use a log sink to store Google Cloud VMware Engine API audit logs. You can route audit logs to different destinations as required. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Collect VMware-level platform logs
| Google control ID | GCVE-CO-4.2 |
|---|---|
| Implementation | Required |
| Description | To collect VMware-level platform logs (for example, vCenter and NSX-T syslog messages), configure Google Cloud VMware Engine private clouds to forward syslog logs to a centralized log aggregator. These logs aren't collected in VMware Engine audit logs and must be collected separately. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Monitor applications using Logging and Monitoring
| Google control ID | GCVE-CO-4.3 |
|---|---|
| Implementation | Required |
| Description | Install the standalone agent to enable Cloud Logging and Cloud Monitoring from the VMware vSphere platform. The standalone agent lets you collect workload-level logs and send them to Logging and Monitoring for analysis and storage. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Create private clouds in regions that match your data-residency requirements
| Google control ID | GCVE-CO-2.1 |
|---|---|
| Implementation | Required |
| Description | Create Google Cloud VMware Engine private clouds in regions that match your organization's data-residency requirements. Private clouds are long-lived provisioned resources that are non-trivial to deploy and relocate. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Implement a backup and disaster recovery strategy
| Google control ID | GCVE-CO-5.1 |
|---|---|
| Implementation | Required |
| Description | Implement Backup and DR Service or a third-party backup solution for workloads. By default, Google Cloud VMware Engine only automatically backs up vCenter and NSX config. Backup and DR makes VM backup and recovery easier. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Implement application-level encryption for VMware workloads
| Google control ID | GCVE-CO-1.1 |
|---|---|
| Implementation | Required |
| Description | Ensure applications that run on Google Cloud VMware Engine encrypt their traffic. VMware Engine doesn't encrypt workload traffic in transit. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Enable data-in-transit encryption on VMware vSAN clusters
| Google control ID | GCVE-CO-2.3 |
|---|---|
| Implementation | Required |
| Description | Enable data-in-transit encryption on VMware vSAN clusters to encrypt data, metadata, and file service traffic. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Configure vSAN data-at-rest encryption to use CMEK
| Google control ID | GCVE-CO-2.2 |
|---|---|
| Implementation | Required |
| Description | If required by your regulatory environment, configure vSAN data-at-rest encryption to use customer-managed encryption keys (CMEKs). |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
Rotate the keys used for vSAN data-at-rest encryption
| Google control ID | GCVE-CO-2.4 |
|---|---|
| Implementation | Required |
| Description | Manage the lifecycle of the key encryption keys (KEKs) that you use for vSan data-at-rest encryption. Implement a key rotation procedure that aligns with your organization's requirements. |
| Applicable products |
|
| Related NIST-800-53 controls |
|
| Related CRI profile controls |
|
| Related information |
What's next
Review data management controls.
See more Google Cloud security best practices and guidelines.