Agent Substrate on GKE is now available for evaluation and non-production use. Production support is offered on an allowlist basis under a limited GA program.

Agent Substrate runs agentic workloads at scale on GKE clusters. To reduce resource usage, Agent Substrate suspends idle agents and takes a snapshot of the agent's active memory (RAM) and local files. When a suspended agent is triggered, the system restores the agent's state onto an available sandbox with sub-second latency.

Agent Substrate improves on the capabilities of Agent Sandbox by bypassing the bottlenecks of the standard Kubernetes control plane to run significantly more concurrent agents per machine.

For more information, see About GKE Substrate.

New features in 1.37

• The PodCertificateRequest and ClusterTrustBundle APIs are generally available and promoted to certificates.k8s.io/v1.
• DRA device metadata publishing is enabled with beta support (for use with DRA drivers which include this feature).
• HorizontalPodAutoscaler enables beta support for scaling to zero.
• StatefulSet maxUnavailable support is now enabled.

GKE support for using the c3-standard-*-lssd machine types as Confidential GKE Nodes with Intel TDX is generally available. For more information, see the following:

• To use this feature with GKE, see Encrypt workload data in-use with Confidential GKE Nodes.
• To learn more about the feature from Compute Engine, see Confidential VM overview.
• To see supported configurations, including machine support, see Supported configurations and its "Machine types, CPUs, and zones" section.

The bare metal machine types from the C3 machine series are now generally available with GKE clusters. You can now provision machine types such as c3-standard-192-metal in Standard mode with any available GKE version.

To use these machine types with Autopilot mode, ComputeClasses, and node pool auto-creation, you must specify the exact machine type using a custom ComputeClass and use GKE version 1.35.3-gke.1389000 or later.

Session affinity support using GCPTrafficDistributionPolicy for GKE Gateway is generally available. This release currently supports single-cluster GKE Gateway load balancers using the following GatewayClasses:

• gke-l7-rilb
• gke-l7-regional-external-managed
• gke-l7-global-external-managed

In addition to the session affinity types available in Preview, you can now use the STRONG_COOKIE_AFFINITY type, which provides the most persistent session stickiness among the session affinity types available in Google Cloud Application Load Balancers.

The session affinity types require the following minimum GKE versions:

• CLIENT_IP, HEADER_FIELD, GENERATED_COOKIE, and HTTP_COOKIE: version 1.35.2-gke.1269001 or later
• STRONG_COOKIE_AFFINITY: version 1.36.3-gke.1767000 or later

For more information, see Configure session affinity using GCPTrafficDistributionPolicy.

Network Endpoint Group (NEG) pre-provisioning is now available in Preview. With this feature, you can force the creation of empty zonal GCE_VM_IP_PORT NEGs in specified zones (or all zones within a region) during Service creation, regardless of whether the cluster has nodes in those zones. By extending the cloud.google.com/neg Service annotation with a custom zones parameter, you can seamlessly automate infrastructure deployments (such as attaching NEGs to backend services) without waiting for workloads to deploy. For more information, see Pre-provisioning empty NEGs.

In GKE version 1.36 and later, GCPAuthzPolicy and GCPAuthzExtension resources for GKE Gateway are now available in Preview. You can use these resources to enforce identity-based access control and zero-trust authorization on incoming traffic at the Gateway layer. These capabilities are supported on the following GatewayClasses:

• gke-l7-global-external-managed
• gke-l7-regional-external-managed
• gke-l7-rilb

For more information, see Configure the GCPAuthzExtension resource.

You can generate optimized GKE configurations that can improve performance for specific workloads, such as Redis and MySQL, by using the gcloud CLI. The configurations are ConfigMaps and ComputeClasses that apply performance recommendations to the workloads and the nodes. These optimizations are available in Preview for GKE version 1.31.1-gke.12000 or later. You can measure the performance improvements by using open source benchmarks. For more information, see Optimize for workloads on GKE.

Feature

In GKE version 1.36.0-gke.3302001 and later, you can run Arm workloads on the Autopilot container-optimized compute platform by using the general-purpose autopilot-arm and autopilot-arm-spot ComputeClasses. You can select these ComputeClasses in Autopilot or Standard clusters. GKE runs the workloads that select these ComputeClasses in Autopilot mode. This compute platform improves Pod scheduling latency, especially during autoscaling operations. For more information, see the following documents:

• Deploy workloads in Autopilot mode
• Autopilot Arm workloads

TPU Subslicing (also known as Dynamic Subslicing) is now generally available for Ironwood (TPU7x). This feature enables you to incrementally provision node pools for a cube or litepod, breaking them into smaller slices (subslices) to run workloads requiring smaller topologies. Updates in this GA release include:

• Dynamic sub-slicing (topologies smaller than 4x4x4, such as 2x2x1, 2x2x2, 2x2x4, and 2x4x4): Supported in GKE version 1.36.0-gke.3712000 or later.
• Dynamic super-slicing (topologies 4x4x4 or larger): Supported in GKE version 1.35.2-gke.1842000 or later.
• Partition Health Labels: The partition state label is updated to cloud.google.com/gke-tpu-partition-[shape]-state to specify smaller subslice shapes. It also introduces UNSET and INCOMPLETE states. Support for the DEGRADED state only applies to the top-level 4x4x4 topology, and not for smaller sub-slicing topologies.

For more information, see About GKE dynamic slicing.

GKE now supports opting out of the default kubernetes.io/arch=arm64:NoSchedule taint on Arm nodes in Standard node pools and in custom ComputeClasses. To opt out of the default taint, set the --node-architecture-taint-behavior gcloud CLI flag to NONE for a node pool or set the taintConfig.architectureTaintBehavior field to NONE for a ComputeClass. By configuring this behavior, you allow workloads that lack explicit Arm tolerations to be scheduled on Arm-based machine families (such as N4A and C4A). This is useful for running multi-architecture workloads or simplifying scheduling in mixed-mode clusters. For more information, see Configure the default Arm architecture taint.

Feature

GKE Gateway and Inference Gateway now support Cross-Origin Resource Sharing (CORS). You can configure a CORS filter directly on an HTTPRoute resource by using the portable syntax standardized by Gateway API. This feature is available in Preview in GKE version 1.35 and later for the following GatewayClasses:

• gke-l7-rilb
• gke-l7-regional-external-managed
• gke-l7-global-external-managed

For more information, see Configure Cross-Origin Resource Sharing.

In GKE version 1.36 and later, GKE Dataplane V2 with NetworkPolicies supports up to 15,000 nodes per cluster, increased from the previous limit of 7,500 nodes. For clusters exceeding 5,000 nodes, contact Cloud Customer Care to request a quota increase. For more information, see Cluster size limits and requirements.

Feature

In version 1.36.2-gke.1498000 and later, GKE supports mixed-protocol Services of type LoadBalancer in general availability (GA). Mixed-protocol Services let both external (NetLB) and internal (ILB) passthrough Network Load Balancers handle simultaneous TCP and UDP traffic on a single IP address across IPv4, IPv6, and dual-stack environments.

In GKE version 1.36.0-gke.3204000 and later, when you manually or automatically create a GKE node pool that consumes capacity reservations, you can stop GKE from falling back to on-demand capacity if reserved capacity isn't available. To consume any matching reservation without fallback, specify the any-reservation-then-fail reservation affinity in your node pool creation request, Pod specification, or ComputeClass specification. In ComputeClasses, this reservation affinity lets GKE move on to the next priority rule instead of creating on-demand compute resources. For more information, see Consuming reserved zonal resources.

Feature

GKE version 1.33 now supports the N4D machine series for node pool auto-creation and Autopilot clusters in the following patch versions and later:

• Node pool auto-creation: 1.33.12-gke.1208000 and later
• Autopilot: 1.33.13-gke.1079000 and later

Feature

In GKE version 1.36.0-gke.4447000 and later, the VerticalPodAutoscaler supports CPU startup boost, which temporarily increases CPU requests during application startup to improve startup latency and cost efficiency. This feature is available in Preview.

Rollout sequencing with custom stages is now generally available. This version of rollout sequencing, which is recommended if you're configuring an environment for the first time, offers a robust set of features including the following:

• Define custom stages: Sequence the rollout of a new GKE version across environments. With custom stages, you can, for example, deploy a new version on a small subset of production clusters before a wider rollout.
• Choose the scope of rollouts: Decide what types of versions that GKE rolls out in the sequence. For example, you can have GKE roll out patch versions, but not minor versions, across a sequence.
• Initiate a rollout: Create a rollout of a specific version, if you want GKE to roll out that version across your sequence.
• Manage a rollout: Pause, resume, cancel rollouts, or complete rollout stages as needed.

For more information, see About rollout sequencing with custom stages.

The network-optimized C4N machine series is available with GKE clusters running 1.36.0-gke.3009002 or later. You can use C4N machine types in Standard or Autopilot mode.

In GKE version 1.34 and later, you can configure Google Cloud Managed Service for Prometheus to collect Pressure Stall Information (PSI) metrics from cAdvisor. You can use PSI metrics to monitor CPU, memory, and I/O congestion and stall times for your containers, Pods, and nodes. For more information, see Collect specific Prometheus metrics from Kubernetes.

Feature

In GKE version 1.35.3-gke.1389000 and later, you can run GPU workloads on Confidential GKE Nodes with certain G4 machine types and NVIDIA RTX PRO 6000 GPUs. This feature is available in Preview. For more information, see Encrypt GPU workload data in use with Confidential GKE Nodes.

Feature

In GKE version 1.36.0-gke.1601000 and later, you can enable the logging of VerticalPodAutoscaler decisions in Cloud Logging. You can use these logs to understand why specific vertical Pod autoscaling decisions were made. This feature is available in Preview. For more information, see Collect vertical Pod autoscaler event logs.

Feature

GKE Gateway now supports backend mutual TLS (mTLS). In addition to backend authenticated TLS, backend mTLS allows the GKE Gateway load balancer to authenticate its identity to backend Pods by presenting a client certificate. GKE Gateway configures backend mTLS using the standard Gateway API spec.tls.backend.clientCertificateRef field.

This feature is supported for the following GatewayClasses:

• gke-l7-global-external-managed
• gke-l7-regional-external-managed
• gke-l7-rilb

For more information, see Configure backend mutual TLS (mTLS) for a Gateway.

GKE Gateway now supports frontend mTLS (client certificate validation). Frontend mTLS allows the Gateway to authenticate client-presented certificates. This feature is available for the following GatewayClasses:

• gke-l7-global-external-managed
• gke-l7-regional-external-managed
• gke-l7-rilb

For more information, see Configure frontend mTLS for a Gateway.

GKE is introducing the following changes to expand the capabilities of maintenance exclusions:

• Per-node pool maintenance exclusions: Available in release channels, these replicate the functionality of disabling node pool auto-upgrades when your cluster isn't enrolled in a release channel.
• Extended "No upgrades" exclusion: The "No upgrades" default maintenance exclusion can now be up to 90 days long.

For more information, see Maintenance exclusions.

GKE Gateway now supports backend authenticated TLS for Gateway-originated connections to Pods or InferencePools for the following GatewayClasses:

• gke-l7-global-external-managed
• gke-l7-regional-external-managed
• gke-l7-rilb

C4A bare metal instances are generally available with GKE clusters. For more information, see the Arm workloads on GKE document, including the "Requirements and limitations" section for specific version requirements.

Feature

Confidential GKE Nodes now support cluster level enablement of AMD SEV-SNP and Intel TDX on GKE Autopilot.

Feature

In GKE versions 1.36.0-gke.2459000 and later, you can directly configure Cloud Logging for L4 load balancer backend services by using the L4LBConfig CustomResourceDefinition (CRD).

This feature is available for the following load balancer types:

• Internal L4 load balancers with subsetting enabled.
• External L4 load balancers with regional backend services (RBS) enabled.

To monitor the efficiency of the GKE training JobSet, the following two GKE system metrics are available in Preview:

• kubernetes.io/jobset/scheduling_goodput: the fraction of time that all the resources required to run the training JobSet are available.
• kubernetes.io/jobset/proxy_runtime_goodput: the fraction of time that all required accelerators are productive. This metric provides an estimate of the real runtime goodput.

For details about GKE metrics, see Kubernetes metrics. For details about goodput metrics that are used to measure efficiency, see Monitor goodput with the ML Goodput Measurement library.

You can also view these new GKE metrics in the JobSet monitoring dashboard.

GKE now supports concurrent node pool upgrades for clusters (Preview). By default, GKE automatically upgrades one node pool at a time. To decrease the total time required to upgrade your cluster, you can now configure the maximum number of node pools that GKE auto-upgrades simultaneously. This feature is supported for both Standard and Autopilot clusters. For more information, see Configure concurrent node pool upgrades.

Feature

Managed OpenTelemetry on GKE now supports the collection of multimodal prompts and responses (Preview) for LangGraph and Agent Development Kit (ADK) agents. You can view and analyze the data in the Trace Explorer and BigQuery platforms. For more details, see Collect multimodal prompts and responses data.

GKE Pod Snapshots is generally available on clusters that run version 1.35.3-gke.1234000 or later. For more information, see About GKE Pod snapshots.

Feature

In GKE Standard clusters, live migration is now supported on Confidential GKE Nodes that use C3D machine series with AMD SEV enabled.

New features in 1.36

• L4 Internal Load Balancer services now use GKE subsetting by default. This change transitions the default implementation from Instance Groups to Network Endpoint Groups (NEGs). This NEG-based approach provides improved scalability and enhanced performance through faster synchronization. This change applies only to newly created Internal Load Balancer services, while existing services remain unaffected and continue using Instance Groups.
• Mutating Admission Policies are now Generally Available. Mutating admission policies allow for resource mutations using Common Expression Language (CEL) expressions as a more efficient alternative to mutating admission webhooks. For more information, see the kubernetes documentation.
• The kube-dns image switches from an implementation based on kubernetes/dns to one based on CoreDNS. This implementation is more efficient, and it supports bigger headless services, more upstream DNS services, and more concurrent connections.

Google Kubernetes Engine now offers support for AI zones. To learn more, see AI zones.

For clusters running GKE version 1.35.3-gke.1389000 or later, you can now use the c4a-highmem-96-metal (Preview) machine type from the C4A machine series with the following features:

• Autopilot mode
• Node auto-provisioning
• ComputeClasses that auto-create node pools
• Cluster autoscaling

Feature

Starting in GKE version 1.35.2-gke.1842000, you can install the Slurm Operator add-on for GKE (Preview). This managed installation of the Slurm Operator allows you to enable Slurm scheduling capabilities on any GKE cluster. The add-on provides the foundation to build customized AI and HPC platforms, including CPU, GPU, and TPU machines (covering specific scenarios). For more information, see About Slurm on GKE.

Accelerator network profile is Generally Available (GA), simplifying your AI/ML node pool setup. Accelerator network profile automates networking configuration, including the creation of necessary VPCs and subnets, removing the need for complex, manual steps previously required for configuring GPU and TPU workloads. For details, see Configure automated networking for accelerator VMs.

Gateway API v1.5 is supported in GKE version 1.35.2-gke.1842000 and later. The GKE Gateway controller passes core conformance tests for this version of the Gateway API.

Feature

GKE managed DRANET is now Generally Available (GA) for GKE version 1.35.2-gke.1842000 or later.

GKE DRANET is a managed feature that implements the Kubernetes Dynamic Resource Allocation (DRA) API for high-performance networking. The GA release expands support beyond the preview phase to include the following hardware:

• NVIDIA GPU Instances: Support for instances starting from A3 Ultra, including A4, A4X, and A4X Max.
• Cloud TPU Instances: Support for TPU v6e and TPU v7x.

For more information, see Allocate network resources by using GKE managed DRANET.

To provide more controls over the control plane version upgrade, you can now do the following:

• Configure a frequency of disruption from auto-upgrades by using the cluster disruption budget. For more information, see Control the frequency of disruption from auto-upgrades.
• Continue using an existing control plane patch for a longer period, which facilitates large-scale upgrade and downgrade operations. For more information, see Patch version support.

In GKE version 1.35 and later, all organization and cluster administrators can granularly control which privileged Autopilot partner workloads can run in GKE clusters. Additionally, approved customers can authorize and run their own privileged workloads in Autopilot mode by using custom allowlists.

For more information, see About Autopilot privileged workloads.

Managed OpenTelemetry for GKE is available in Preview for clusters running version 1.34.1-gke.2178000 or later. Managed OpenTelemetry for GKE provides a fully managed and simplified experience for collecting OpenTelemetry Protocol (OTLP) traces, metrics, and logs on GKE. This feature includes the following characteristics:

• Managed collection: an in-cluster OTLP endpoint that automatically routes telemetry to the Cloud Telemetry API.
• Automatic configuration: a new Instrumentation custom resource that automatically injects environment variables into your workloads to simplify OTLP ingestion.

For more information, see Managed OpenTelemetry for GKE.