Infrastructure controls

This document includes the best practices and guidelines for Google Cloud services such as Compute Engine, Google Kubernetes Engine (GKE), Pub/Sub, Dataflow, and Cloud Run functions when running workloads on Google Cloud.

Compute controls

These controls apply to compute services.

Define VM instances that can enable IP forwarding

Google control ID VPC-CO-6.3
Implementation Required
Description
The compute.vmCanIpForward constraint defines the VM instances that can enable IP forwarding. By default, any VM can enable IP forwarding in any virtual network. Specify VM instances using one of the following formats:
  • under:organizations/ORGANIZATION_ID
  • under:folders/FOLDER_ID
  • under:projects/PROJECT_ID
  • projects/PROJECT_ID/zones/ZONE/instances/INSTANCE_NAME.
This constraint isn't retroactive.
Applicable products
  • Organization Policy Service
  • Virtual Private Cloud (VPC)
  • Compute Engine
Path constraints/compute.vmCanIpForward
Operator =
Value
  • Your list of VM instances that can enable IP forwarding.
Type List
Related NIST-800-53 controls
  • SC-7
  • SC-8
Related CRI profile controls
  • PR.AC-5.1
  • PR.AC-5.2
  • PR.DS-2.1
  • PR.DS-2.2
  • PR.DS-5.1
  • PR.PT-4.1
  • DE.CM-1.1
  • DE.CM-1.2
  • DE.CM-1.3
  • DE.CM-1.4
Related information

Disable VM-nested virtualization

Google control ID VPC-CO-6.6
Implementation Required
Description
The compute.disableNestedVirtualization boolean constraint disables hardware-accelerated nested virtualization for Compute Engine VMs.
Applicable products
  • Organization Policy Service
  • Virtual Private Cloud (VPC)
  • Compute Engine
Path constraints/compute.disableNestedVirtualization
Operator Is
Value
  • True
Type Boolean
Related NIST-800-53 controls
  • SC-7
  • SC-8
Related CRI profile controls
  • PR.AC-5.1
  • PR.AC-5.2
  • PR.DS-2.1
  • PR.DS-2.2
  • PR.DS-5.1
  • PR.PT-4.1
  • DE.CM-1.1
  • DE.CM-1.2
  • DE.CM-1.3
  • DE.CM-1.4
Related information

Restrict external IP addresses on VMs

Google control ID VPC-CO-6.2
Implementation Required
Description

Unless needed, prevent the creation of Compute Engine instances with public IP addresses. The compute.vmExternalIpAccess list constraint defines the set of Compute Engine VM instances that can have external IP addresses.

Prevent Compute Engine instances from having external IP addresses to drastically reduce their exposure to the internet. Any instance with an external IP address is immediately discoverable and becomes a direct target for automated scans, brute-force attacks, and attempts to exploit vulnerabilities. Instead, require instances to use private IP addresses and manage access through controlled, authenticated, and logged pathways like the Identity-Aware Proxy (IAP) tunnel or a bastion host.

Adopting this deny-by-default posture is a foundational security best practice that helps minimize your attack surface and enforces a zero-trust approach to your network. This constraint isn't retroactive.
Applicable products
  • Organization Policy Service
  • Virtual Private Cloud (VPC)
  • Compute Engine
Path constraints/compute.vmExternalIpAccess
Operator =
Value
  • The list of VM instances in your organization that can have external IP addresses.
Type List
Related NIST-800-53 controls
  • SC-7
  • SC-8
Related CRI profile controls
  • PR.AC-5.1
  • PR.AC-5.2
  • PR.DS-2.1
  • PR.DS-2.2
  • PR.DS-5.1
  • PR.PT-4.1
  • DE.CM-1.1
  • DE.CM-1.2
  • DE.CM-1.3
  • DE.CM-1.4
Related information

Define permitted external IP addresses for VM instances

Google control ID CBD-CO-6.3
Implementation Required
Description

The compute.vmExternalIpAccess list constraint lets you restrict external access to virtual machines by not assigning external IP addresses. Configure this list constraint to deny all external IP addresses to virtual machines.
Applicable products
  • Organization Policy Service
  • Compute Engine
Path compute.vmExternalIpAccess
Operator =
Value
  • Deny All
Type List
Related NIST-800-53 controls
  • AC-3
  • AC-12
  • AC-17
  • AC-20
Related CRI profile controls
  • PR.AC-3.1
  • PR.AC-3.2
  • PR.AC-4.1
  • PR.AC-4.2
  • PR.AC-4.3
  • PR.AC-6.1
  • PR.AC-7.1
  • PR.AC-7.2
  • PR.PT-3.1
  • PR-PT-4.1
Related information

Require VPC connector for Cloud Run functions

Google control ID CF-CO-4.4
Implementation Required
Description

The cloudfunctions.requireVPCConnector boolean constraint requires that administrators specify a Serverless VPC Access connector when they deploy a Cloud Run function. When enforced, functions must specify a connector.
Applicable products
  • Organization Policy Service
  • Cloud Run functions
Path constraints/cloudfunctions.requireVPCConnector
Operator =
Value
  • True
Type Boolean
Related NIST-800-53 controls
  • SC-7
  • SC-8
Related CRI profile controls
  • PR.AC-5.1
  • PR.AC-5.2
  • PR.DS-2.1
  • PR.DS-2.2
  • PR.DS-5.1
  • PR.PT-4.1
  • DE.CM-1.1
  • DE.CM-1.2
  • DE.CM-1.3
  • DE.CM-1.4
Related information

Configure message storage policies

Google control ID PS-CO-4.1
Implementation Optional
Description
If you publish messages to the global Pub/Sub endpoint, Pub/Sub automatically stores the messages in the nearest Google Cloud region. To control which regions your messages are stored in, configure a message storage policy on your topic. Use one of the following ways to configure message storage policies for topics:
  • Set a message storage policy using the Resource Location Restriction (gcp.resourceLocations) organization policy constraint.
  • Configure a message storage policy when creating a topic. For example:

    gcloud pubsub topics create TOPIC_ID \--message-storage-policy-allowed-regions=REGION1, REGION2
Applicable products
  • Organization Policy Service
  • Pub/Sub
Related NIST-800-53 controls
  • AC-3
  • AC-17
  • AC-20
Related CRI profile controls
  • PR.AC-3.1
  • PR.AC-3.2
  • PR.AC-4.1
  • PR.AC-4.2
  • PR.AC-4.3
  • PR.AC-6.1
  • PR.PT-3.1
  • PR.PT-4.1
Related information

Turn off external IP addresses for Dataflow jobs

Google control ID DF-CO-6.1
Implementation Optional
Description

Turn off external IP addresses for administrative and monitoring tasks that are related to Dataflow jobs. Instead, configure access to your Dataflow worker VMs using SSH.

Enable Private Google Access and specify one of the following options in your Dataflow job:

  • --usePublicIps=false and --network=NETWORK-NAME
  • --subnetwork=SUBNETWORK-NAME

Where:

  • NETWORK-NAME: The name of your Compute Engine network.
  • SUBNETWORK-NAME: The name of your Compute Engine subnetwork.
Applicable products
  • Compute Engine
  • Dataflow
Related NIST-800-53 controls
  • SC-7
  • SC-8
Related CRI profile controls
  • PR.AC-5.1
  • PR.AC-5.2
  • PR.DS-2.1
  • PR.DS-2.2
  • PR.DS-5.1
  • PR.PT-4.1
  • DE.CM-1.1
  • DE.CM-1.2
  • DE.CM-1.3
  • DE.CM-1.4
Related information

Use network tags for firewall rules

Google control ID DF-CO-6.2
Implementation Optional
Description

Network tags are text attributes that attach to Compute Engine VMs such as Dataflow worker VMs. Network tags let you make VPC network firewall rules and some custom static routes applicable to specific VM instances. Dataflow supports adding network tags to all worker VMs that run a particular Dataflow job.
Applicable products
  • Compute Engine
  • Dataflow
Related NIST-800-53 controls
  • SC-7
  • SC-8
Related CRI profile controls
  • PR.AC-5.1
  • PR.AC-5.2
  • PR.DS-2.1
  • PR.DS-2.2
  • PR.DS-5.1
  • PR.PT-4.1
  • DE.CM-1.1
  • DE.CM-1.2
  • DE.CM-1.3
  • DE.CM-1.4
Related information

Container controls

These controls apply to containers within GKE.

Restrict control plane access

Google control ID GKE-CO-1.1
Implementation Required
Description

By default, the Google Kubernetes Engine (GKE) cluster control plane and nodes have internet routable addresses that can be accessed from any IP address. Restrict network access to the control plane by using a DNS-based endpoint and by creating private clusters. The control plane is the management center for a Kubernetes cluster, and exposing it to the internet makes it a prime target for attackers. This configuration makes the control plane private and removes it from the internet.

Restricting control plane access helps ensure that only trusted devices within your organization's private network can manage the cluster, drastically reducing the risk of an external attack.
Applicable products
  • GKE
Related NIST-800-53 controls
  • SC-7
Related CRI profile controls
  • PR.AC-3.1
Related information

Use least-privilege firewall rules

Google control ID GKE-CO-1.2
Implementation Required
Description

When you create firewall rules, use the principle of least privilege to provide access only for the required purpose. Ensure that your firewall rules don't conflict with the GKE default firewall rules when possible.
Applicable products
  • GKE
Related NIST-800-53 controls
  • AC-3
  • AC-4
  • AC-17
  • SC-7
Related information

Use Google Groups for RBAC

Google control ID GKE-CO-1.3
Implementation Required
Description

Use Google Groups for role-based access control (RBAC), which also lets you integrate with your existing user account management practices, such as revoking access when someone leaves your organization. Google Groups for RBAC helps provide efficient management of cluster access using Identity and Access Management (IAM) and Google Groups, which is suitable for most organizations that use Google Groups.
Applicable products
  • Google Kubernetes Engine (GKE)
Related NIST-800-53 controls
  • AC-2
Related CRI profile controls
  • PR.AC-1.1
Related information

Enable Shielded GKE Nodes

Google control ID GKE-CO-1.4
Implementation Required
Description

Enable Shielded GKE Nodes for cryptographic verification of the nodes in your cluster. Shielded GKE Nodes provide strong, verifiable node identity and integrity. Enable Shielded GKE Nodes when creating or updating clusters. Where possible, use Shielded GKE Nodes with secure boot to also authenticate the boot components of your node VMs during the boot process. Don't use secure boot if you need third-party unsigned kernel modules.

Shielded GKE Nodes is enabled by default when you create a cluster.
Applicable products
  • GKE
Related NIST-800-53 controls
  • AC-3
  • AC-4
  • AC-17
  • SC-7
Related information

Use Container-Optimized OS with containerd runtime

Google control ID GKE-CO-1.5
Implementation Required
Description

Use Container-Optimized OS to implement a hardened and managed container OS. General-purpose operating systems include many extra programs that aren't needed to run containers and therefore create a larger, unnecessary target for attackers. Container-Optimized OS is a minimal, locked-down operating system that significantly reduces this attack surface by including only what is necessary. As a managed OS, Container-Optimized OS also has security patches that are automatically applied by Google, which help ensure critical vulnerabilities are fixed and reduces your operational workload.

An image that includes Container-Optimized OS with containerd (cos_containerd) has containerd as the main container runtime directly integrated with Kubernetes. containerd is the core runtime component of Docker and is designed to deliver core container functionality for the Kubernetes Container Runtime Interface (CRI). It is significantly less complex than the full Docker daemon, and therefore has a smaller attack surface.
Applicable products
  • Container-Optimized OS
Related NIST-800-53 controls
  • CM-7
  • SC-7
  • SC-38
  • SI-2
  • SI-7
Related CRI profile controls
  • PR.PT-3.1
Related information

Use Workload Identity Federation for GKE

Google control ID GKE-CO-1.6
Implementation Required
Description

Use Workload Identity Federation for GKE to securely authenticate to Google Cloud APIs from Google Kubernetes Engine (GKE) workloads. Workload Identity Federation for GKE provides a simpler and safer alternative to using service account keys.
Applicable products
  • GKE
Related NIST-800-53 controls
  • IA-2
Related CRI profile controls
  • PR.AC-1.1
Related information

Enable GKE Sandbox

Google control ID GKE-CO-1.7
Implementation Required
Description

Use GKE Sandbox to provide an extra layer of security to help prevent untrusted code from affecting the host kernel on your Google Kubernetes Engine (GKE) cluster nodes. GKE Sandbox enhances workload isolation for untrusted or sensitive workloads, providing an additional layer of protection against container escape attacks.
Applicable products
  • GKE
Related NIST-800-53 controls
  • SC-39
Related CRI profile controls
  • PR.DS-1.1
Related information

Disable the kubelet read-only port

Google control ID GKE-CO-1.9
Implementation Required
Description

Disable kubelet read-only port 10255 and use the more secure port 10250 instead. Kubernetes doesn't perform any authentication or authorization checks on this port. The kubelet serves the same endpoints on the more secure, authenticated port 10250.

You can only disable the insecure kubelet read-only port in GKE version 1.26.4-gke.500 or later.
Applicable products
  • GKE
Related NIST-800-53 controls
  • AC-3
  • SC-7
Related information

Use namespace and RBAC to restrict access to cluster resources

Google control ID GKE-CO-1.10
Implementation Required
Description

Create separate namespaces or clusters for each team and environment to implement least-privilege access to Kubernetes. Assign cost centers and appropriate labels to each namespace for accountability and chargeback. Only give developers the level of access to their namespace that they need to deploy and manage their application, especially in production. Map out the tasks that your users need to undertake against the cluster and define the permissions that they require to do each task.

Assign the appropriate Identity and Access Management (IAM) roles for Google Kubernetes Engine (GKE) to groups and users to provide permissions at the project level and use RBAC to grant permissions on a cluster and namespace level.
Applicable products
  • GKE
  • IAM
Related NIST-800-53 controls
  • AC-3
  • AC-4
  • AC-6
Related information

Restrict traffic among pods

Google control ID GKE-CO-1.11
Implementation Required
Description

By default, all pods in a cluster can communicate with each other. Control pod-to-pod communication as needed for your workloads. Restricting network access to services makes it much more difficult for attackers to move laterally within your cluster, and also offers services some protection against accidental or deliberate denial of service.

There are two ways to control traffic:

  • Use Cloud Service Mesh or Istio for load balancing, service authorization, throttling, quota, metrics and more.
  • Use Kubernetes network policies. Choose this option if you're looking for the basic access control functionality that's exposed by Kubernetes.
Applicable products
  • GKE
Related NIST-800-53 controls
  • AC-3
  • AC-4
  • SC-7
Related information

Use admission controllers to enforce policies

Google control ID GKE-CO-1.12
Implementation Required
Description

Admission controllers are plugins that govern and enforce how the cluster is used. Enable them to use some of the more advanced security features of Kubernetes because they are an important part of the defense in depth approach to hardening your cluster. By default, pods in Kubernetes can operate with capabilities beyond what they require. Use admission controls to restrict the pod's capabilities to only those that are required for that workload.

GKE supports numerous controls for restricting your pods to execute with capabilities that are explicitly granted. For example, Policy Controller is available for clusters in fleets. Kubernetes also has the built-in PodSecurity admission controller that lets you enforce the Pod Security Standards in individual clusters. Policy Controller is a feature of GKE that lets you enforce and validate security on GKE clusters at scale by using declarative policies.
Applicable products
  • GKE
Related NIST-800-53 controls
  • CM-2
  • CM-3
  • CM-6
  • CM-7
Related information

Restrict the ability for workloads to self-modify

Google control ID GKE-CO-1.13
Implementation Required
Description

Certain Kubernetes workloads, especially system workloads, have permission to self-modify. For example, some workloads vertically autoscale themselves. While convenient, self-modification can allow an attacker who has already compromised a node to escalate further in the cluster. For example, an attacker could have a workload on the node change itself to run as a more privileged service account that exists in the same namespace.

Don't grant workloads the permission to modify themselves by default. When self-modification is necessary, limit permissions by applying Gatekeeper or Policy Controller constraints, such as NoUpdateServiceAccount from the open source Gatekeeper library, which provides several useful security policies.

When you deploy policies, allow the controllers that manage the cluster lifecycle to bypass the policies. Controllers must make changes to the cluster, such as applying cluster upgrades. For example, if you deploy the NoUpdateServiceAccount policy on GKE, you must set the following parameters in the constraint:

parameters: allowedGroups: - system:masters allowedUsers: - system:addon-manager
Applicable products
  • GKE
Related NIST-800-53 controls
  • AC-3
  • CM-3
  • SI-7
Related information

Monitor your cluster configurations

Google control ID GKE-CO-1.14
Implementation Required
Description

Audit your cluster configurations for deviations from your defined settings. Settings covered in these best practices, as well as other common misconfigurations, can be automatically checked using Security Command Center.
Applicable products
  • GKE
  • Security Command Center
Related NIST-800-53 controls
  • SI-4
  • SI-5
  • RA-5
Related information

Enforce Binary Authorization

Google control ID BIN-CO-1.1
Implementation Required
Description

Use Binary Authorization to make sure trusted images are deployed to Google Kubernetes Engine (GKE) and Cloud Run. Binary Authorization helps ensure that only verified and trusted container images can be deployed in your clusters, strengthening software supply chain security.
Applicable products
  • GKE
  • Binary Authorization
  • Cloud Run
Path constraints/binaryauthorization.requireBinauthz
Operator ==
Value
  • True
Related NIST-800-53 controls
  • SI-7
Related CRI profile controls
  • PR.DS-6.1
Related information

What's next