This document introduces the concepts that you need to understand
how to configure an external Application Load Balancer.
An external Application Load Balancer is a proxy-based Layer 7 load balancer that enables you to run
and scale your services behind a single external IP address. The external Application Load Balancer
distributes HTTP and HTTPS traffic to backends hosted on a variety of
Google Cloud platforms (such as Compute Engine,
Google Kubernetes Engine (GKE), and Cloud Storage), as well as external
backends connected over the internet or through hybrid connectivity.
For more information, see [Application Load Balancer overview: Use
cases](https://docs.cloud.google.com/load-balancing/docs/application-load-balancer#use-cases).
## Modes of operation
You can configure an external Application Load Balancer in the following modes:
- **Global external Application Load Balancer** . This is a global load balancer that is
implemented as a managed service on [Google Front Ends
(GFEs)](https://docs.cloud.google.com/docs/security/infrastructure/design#google-frontend-service). It
uses the open-source [Envoy proxy](https://www.envoyproxy.io/)
to support [advanced traffic
management](https://docs.cloud.google.com/load-balancing/docs/https/traffic-management-global)
capabilities such as traffic mirroring, weight-based traffic splitting, and
request-based or response-based header transformations.
- **Classic Application Load Balancer** . This is the classic external Application Load Balancer that is
global in Premium Tier but can be configured to be regional in Standard
Tier. This load balancer is implemented on [Google Front Ends
(GFEs)](https://docs.cloud.google.com/docs/security/infrastructure/design#google-frontend-service). GFEs
are distributed globally and operate together using Google's global network
and control plane.
- **Regional external Application Load Balancer** . This is a regional load balancer that is
implemented as a managed service on the open-source [Envoy
proxy](https://www.envoyproxy.io/). It includes
[advanced traffic
management](https://docs.cloud.google.com/load-balancing/docs/https/traffic-management-regional)
capabilities such as traffic mirroring, weight-based traffic splitting, and
request-based or response-based header transformations.
| Load balancer mode | Recommended use cases | Capabilities |
|---|---|---|
| Global external Application Load Balancer | Use this load balancer for external HTTP(S) workloads with globally dispersed users or backend services in multiple regions. | - Compatible with GKE using [Gateway](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/gateway-api.md.txt#gatewayclass) (fully orchestrated) or [Standalone NEGs](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/standalone-neg.md.txt) (manual orchestration) - Supports [advanced traffic management](https://docs.cloud.google.com/load-balancing/docs/https/traffic-management-global.md.txt) - [Global Anycast external IP addresses over Premium Tier](https://docs.cloud.google.com/network-tiers#tab1) - Can access backends across multiple regions - Supports [Cloud CDN](https://docs.cloud.google.com/cdn) - Supports [Cloud Armor](https://docs.cloud.google.com/armor) |
| Classic Application Load Balancer | This load balancer is global in Premium Tier. In the [Premium Network Service Tier](https://docs.cloud.google.com/network-tiers/docs/overview.md.txt), this load balancer offers multi-region load balancing, attempts to direct traffic to the closest healthy backend that has capacity, and terminates HTTP(S) traffic as close as possible to your users. For more information about the request distribution process, see [Traffic distribution](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#load_distribution_algorithm). In the [Standard Network Service Tier](https://docs.cloud.google.com/network-tiers/docs/overview.md.txt), this load balancer can distribute traffic to backends in a single region only. | - Compatible with GKE using [Gateway](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/gateway-api.md.txt#gatewayclass) (fully orchestrated), [Ingress](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/ingress-xlb) (fully orchestrated), or [Standalone NEGs](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/standalone-neg.md.txt) (manual orchestration) - Supports Google Cloud Armor - Fewer traffic routing features See the [Load balancing features](https://docs.cloud.google.com/load-balancing/docs/features.md.txt) page for the full list of capabilities. |
| Regional external Application Load Balancer | This load balancer contains many of the features of the existing classic Application Load Balancer, along with [advanced traffic management](https://docs.cloud.google.com/load-balancing/docs/https/traffic-management-regional.md.txt) capabilities. Use this load balancer if you want to serve content from only one geolocation (for example, to meet compliance regulations). This load balancer can be configured in either Premium or Standard Tier. | - Compatible with GKE using [Standalone NEGs](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/standalone-neg.md.txt) (manual orchestration) - Supports advanced traffic management capabilities - [Regional VIPs using Standard Network Tier](https://docs.cloud.google.com/network-tiers#tab2) - Terminates TLS in a single region that you configure - Serves content from the configured region only For the complete list, see [Load balancing features](https://docs.cloud.google.com/load-balancing/docs/features.md.txt). |
### Identify the mode
### Console
1. In the Google Cloud console, go to the **Load balancing** page.
[Go to Load balancing](https://console.cloud.google.com/networking/loadbalancing)
2. In the **Load Balancers** tab, the load balancer type, protocol, and
region are displayed. If the region is blank, then the load balancer is
global. The following table summarizes how to identify the mode of the
load balancer.
| Load balancer mode | Load balancer type | Access type | Region |
|---|---|---|---|
| Global external Application Load Balancer | Application | External | |
| Classic Application Load Balancer | Application (Classic) | External | |
| Regional external Application Load Balancer | Application | External | Specifies a region |
### gcloud
To determine the mode of a load balancer, run the following command:
```
gcloud compute forwarding-rules describe FORWARDING_RULE_NAME
```
In the command output, check the load balancing scheme, region, and network
tier. The following table summarizes how to identify the mode of the load
balancer.
| Load balancer mode | Load balancing scheme | Forwarding rule | Network tier |
|---|---|---|---|
| Global external Application Load Balancer | EXTERNAL_MANAGED | Global | Premium |
| Classic Application Load Balancer | EXTERNAL | Global | Standard or Premium |
| Regional external Application Load Balancer | EXTERNAL_MANAGED | Specifies a region | Standard or Premium |
> [!IMPORTANT]
> **Important:** After you create a load balancer, you can't edit its mode. Instead, you must delete the load balancer and create a new one.
## Architecture
The following resources are required for an external Application Load Balancer
deployment:
- *For regional external Application Load Balancers only* , a [**proxy-only
subnet**](https://docs.cloud.google.com/load-balancing/docs/https#proxy-only-subnet) is used to send connections from the load
balancer to the backends.
- An [**external forwarding rule**](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#forwarding-rule) specifies an external IP
address, port, and target HTTP(S) proxy. Clients use the IP address and port
to connect to the load balancer.
- A [**target HTTP(S) proxy**](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#target-proxies) receives a request from the
client. The HTTP(S) proxy evaluates the request by using the URL map to make
traffic routing decisions. The proxy can also authenticate communications by
using SSL certificates.
- For HTTPS load balancing, the target HTTPS proxy uses [**SSL
certificates**](https://docs.cloud.google.com/load-balancing/docs/https#ssl-certificates) to prove its identity to clients. A target HTTPS proxy supports up to the [documented
number](https://docs.cloud.google.com/load-balancing/docs/quotas#target_pools_and_target_proxies) of SSL certificates.
- The HTTP(S) proxy uses a [**URL map**](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#url-maps) to make a routing
determination based on HTTP attributes (such as the request path, cookies,
or headers). Based on the routing decision, the proxy forwards client
requests to specific backend services or backend buckets. The URL map can
specify additional actions, such as sending redirects to clients.
- A [**backend service**](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#backend-service) distributes requests to healthy
[backends](https://docs.cloud.google.com/load-balancing/docs/features.md.txt#backends). The
global external Application Load Balancers also support **backend buckets** . One or more
**backends** must be connected to the backend service or backend bucket.
- A [**health check**](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#health-checks) periodically monitors the readiness of
your backends. This reduces the risk that requests might be sent to backends
that can't service the request.
- [**Firewall rules**](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#firewall-rules) for your backends to accept health
check probes. Regional external Application Load Balancers require an additional firewall
rule to allow traffic from the proxy-only subnet to reach the backends.
Global
: This diagram shows the components of a global external Application Load Balancer deployment.
This architecture applies to both, the global external Application Load Balancer, and the
classic Application Load Balancer in Premium Tier.
[](https://docs.cloud.google.com/static/load-balancing/images/global-ext-https-lb.svg) Global external Application Load Balancer components (click to enlarge).
Regional
: This diagram shows the components of a regional external Application Load Balancer deployment.
[](https://docs.cloud.google.com/static/load-balancing/images/regional-ext-https-lb.svg) Regional external Application Load Balancer components (click to enlarge).
### Proxy-only subnet
***Proxy-only subnets are only required for regional external Application Load Balancers.***
The proxy-only subnet provides a set of IP addresses that Google uses to run
Envoy proxies on your behalf. You must create *one* proxy-only subnet in each
region of a VPC network where you use regional external Application Load Balancers.
The `--purpose` flag for this proxy-only subnet is set to
`REGIONAL_MANAGED_PROXY`. All [regional Envoy-based load
balancers](https://docs.cloud.google.com/load-balancing/docs/proxy-only-subnets#envoy-lb) in the same region
and VPC network share a pool of Envoy proxies from the same
proxy-only subnet. Further:
- Proxy-only subnets are *only* used for Envoy proxies, not your backends.
- Backend VMs or endpoints of all regional external Application Load Balancers in a region and VPC network receive connections from the proxy-only subnet.
- The IP address of the regional external Application Load Balancer is *not* located in the proxy-only subnet. The load balancer's IP address is defined by its external managed forwarding rule, which is described below.
- Proxy-only subnets can be configured with a stack type of `IPV4_IPV6` or `IPV4_ONLY`.
If you previously created a proxy-only subnet with
`--purpose=INTERNAL_HTTPS_LOAD_BALANCER`, [migrate the subnet's
purpose](https://docs.cloud.google.com/load-balancing/docs/proxy-only-subnets#migrate-purpose) to
`REGIONAL_MANAGED_PROXY` before you can create other Envoy-based load balancers
in the same region of the VPC
network.
### Forwarding rules and IP addresses
[Forwarding rules](https://docs.cloud.google.com/load-balancing/docs/forwarding-rule-concepts.md.txt) route traffic
by IP address, port, and protocol to a load balancing configuration consisting
of a target proxy, URL map, and one or more backend services.
**IP address specification**. Each forwarding rule provides a single IP address
that can be used in DNS records for your application. No DNS-based load
balancing is required. You can either specify the IPv4 or IPv6 address to be
used or let Cloud Load Balancing assign one for you.
**Port specification** . Each forwarding rule for an Application Load Balancer can
reference a [single port from
1-65535](https://docs.cloud.google.com/load-balancing/docs/forwarding-rule-concepts#port_specifications). To
support multiple ports, you must configure multiple forwarding rules. You can
configure multiple forwarding rules to use the same external IP address (VIP)
and to reference the same target HTTP(S) proxy as long as the overall
*combination* of IP address, port, and protocol is unique for each forwarding
rule. This way, you can use a single load balancer with a shared URL map as a
proxy for multiple applications.
The type of forwarding rule, IP address, and load balancing scheme used by
external Application Load Balancers depends on the mode of the load balancer and which [Network
Service Tier](https://docs.cloud.google.com/network-tiers/docs/overview) the load balancer is in.
| Load balancer mode | Network Service Tier | Forwarding rule, IP address, and load balancing scheme | Routing from the internet to the load balancer frontend |
|---|---|---|---|
| Global external Application Load Balancer | Premium Tier | [Global external forwarding rule](https://docs.cloud.google.com/compute/docs/reference/rest/v1/globalForwardingRules/insert.md.txt) [Global external IP address](https://docs.cloud.google.com/compute/docs/reference/rest/v1/globalAddresses/insert.md.txt) Load balancing scheme: `EXTERNAL_MANAGED` | Requests routed to the GFE that is closest to the client on the internet. |
| Classic Application Load Balancer | Premium Tier | [Global external forwarding rule](https://docs.cloud.google.com/compute/docs/reference/rest/v1/globalForwardingRules/insert.md.txt) [Global external IP address](https://docs.cloud.google.com/compute/docs/reference/rest/v1/globalAddresses/insert.md.txt) Load balancing scheme: `EXTERNAL` ^1^ | Requests routed to the GFE that is closest to the client on the internet. |
| Classic Application Load Balancer | Standard Tier | [Regional external forwarding rule](https://docs.cloud.google.com/compute/docs/reference/rest/v1/forwardingRules/insert.md.txt) [Regional external IP address](https://docs.cloud.google.com/compute/docs/reference/rest/v1/addresses/insert.md.txt) Load balancing scheme: `EXTERNAL`^1^ | Requests routed to a GFE in the load balancer's region. |
| Regional external Application Load Balancer | Premium Tier or Standard Tier | [Regional external forwarding rule](https://docs.cloud.google.com/compute/docs/reference/rest/v1/forwardingRules/insert.md.txt) [Regional external IP address](https://docs.cloud.google.com/compute/docs/reference/rest/v1/addresses/insert.md.txt) Load balancing scheme: `EXTERNAL_MANAGED` | Requests reach Google Cloud at the [PoP](https://docs.cloud.google.com/load-balancing/docs/locations.md.txt) closest to the client. Requests are then routed over Google Cloud's premium backbone until they reach Envoy proxies in the same region as the load balancer. |
^1^ It is possible to attach `EXTERNAL_MANAGED` backend services to `EXTERNAL` forwarding rules. However, `EXTERNAL` backend services cannot be attached to `EXTERNAL_MANAGED` forwarding rules. To take advantage of [new features available
only with the global external Application Load Balancer](https://docs.cloud.google.com/load-balancing/docs/https#load-balancer-mode), we recommend that you migrate your existing `EXTERNAL` resources to `EXTERNAL_MANAGED` by using the migration process described at [Migrate
resources from classic to global external Application Load Balancer](https://docs.cloud.google.com/load-balancing/docs/https/migrate-from-classic-global).
For the complete list of protocols supported by external Application Load Balancer
forwarding rules in each mode, see [Load balancer
features](https://docs.cloud.google.com/load-balancing/docs/features#protocols_from_the_clients_to_the_load_balancer).
#### Forwarding rules and VPC networks
This section describes how forwarding rules used by external Application Load Balancers are
associated with VPC networks.
| Load balancer mode | VPC network association |
|---|---|
| Global external Application Load Balancer Classic Application Load Balancer | No associated VPC network. The forwarding rule always uses an [IP address](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#fw-rule-ip-address-table) that is outside the VPC network. Therefore, the forwarding rule has no associated VPC network. |
| Regional external Application Load Balancer | The forwarding rule's VPC network is the network where the proxy-only subnet has been created. You specify the network when you create the forwarding rule. Depending on whether you use an IPv4 address or an IPv6 address range, there is always an explicit or implicit VPC network associated with the forwarding rule. - Regional external IPv4 addresses always exist outside of VPC networks. However, when you create the forwarding rule, you're required to specify the VPC network where the proxy-only subnet has been created. Therefore, the forwarding rule has an explicit network association. - Regional external IPv6 address ranges always exist inside a VPC network. When you create the forwarding rule, you're required to specify the subnet from which the IP address range is taken. This subnet must be in the same region and VPC network where a proxy-only subnet has been created. Thus, there is an implied network association. **Network and subnet requirements.** For IPv6 traffic, your network and subnets must meet the following configuration requirements: - VPC network: you must use a custom mode VPC network configured with the `--enable-ula-internal-ipv6` flag. - Forwarding rule subnet: this subnet must be a dual-stack (`IPv4_IPv6`) or `IPV6_ONLY` subnet with the `ipv6-access-type` set to `EXTERNAL`. **IPv6 address allocation options.** The forwarding rule must reference a `/96` range of IPv6 addresses from the subnet's `/64` external IPv6 address range. The load balancer accepts traffic sent to any address within the allocated `/96` range. To assign this `/96` IPv6 address range, you can use one of the following methods: - Specifying a [reserved external IPv6 address](https://docs.cloud.google.com/vpc/docs/reserve-static-external-ip-address.md.txt#reserve_new_static), either automatically allocated or a custom `/96` range. - Specifying a custom ephemeral IPv6 address. - Letting Google Cloud automatically assign an ephemeral IPv6 address. **Limitations.** To specify a custom ephemeral IPv6 address, you must use the Google Cloud CLI or the API. The Google Cloud console doesn't support specifying custom ephemeral IPv6 addresses for forwarding rules. To deploy a regional external Application Load Balancer with an IPv6 address on GKE, see [Deploy a regional external Gateway](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/deploying-gateways.md.txt#deploy_a_regional_external_gateway) in the GKE documentation. |
### Target proxies
[Target proxies](https://docs.cloud.google.com/load-balancing/docs/target-proxies.md.txt) terminate HTTP(S)
connections from clients. One or more forwarding rules direct traffic to the
target proxy, and the target proxy consults the URL map to determine how to
route traffic to backends.
Do not rely on the proxy to preserve the case of request or response header
names. For example, a `Server: Apache/1.0` response header might appear at the
client as `server: Apache/1.0`.
> [!NOTE]
> **Note:** For internet Network Endpoint Groups (NEGs), requests from the load balancer come from different IP ranges depending on the type of NEG (global or regional). For more information, see [Authenticating
> requests](https://docs.cloud.google.com/load-balancing/docs/negs/internet-neg-concepts#authenticating_requests).
The following table specifies the type of target proxy required by
external Application Load Balancers.
| Load balancer mode | Target proxy types | Proxy-added headers | [Custom headers](https://docs.cloud.google.com/load-balancing/docs/https/custom-headers.md.txt) supported |
|---|---|---|---|
| Global external Application Load Balancer | [Global HTTP](https://docs.cloud.google.com/compute/docs/reference/rest/v1/targetHttpProxies.md.txt), [Global HTTPS](https://docs.cloud.google.com/compute/docs/reference/rest/v1/targetHttpsProxies.md.txt) | The proxies set HTTP request/response headers as follows: - `Via: 1.1 google` (requests and responses) - `X-Forwarded-Proto: [http | https]` (requests only) - `X-Forwarded-For: [SUPPLIED_VALUE,]CLIENT_IP,LOAD_BALANCER_IP` (see [X-Forwarded-For header](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#x-forwarded-for_header)) (requests only) The proxies also set the `X-Cloud-Trace-Context` header if it is not already present. | Yes [Configured on the backend service or backend bucket](https://docs.cloud.google.com/load-balancing/docs/https/custom-headers.md.txt) Not supported with Cloud CDN |
| Classic Application Load Balancer | [Global HTTP](https://docs.cloud.google.com/compute/docs/reference/rest/v1/targetHttpProxies.md.txt), [Global HTTPS](https://docs.cloud.google.com/compute/docs/reference/rest/v1/targetHttpsProxies.md.txt) | The proxies set HTTP request/response headers as follows: - `Via: 1.1 google` (requests and responses) - `X-Forwarded-Proto: [http | https]` (requests only) - `X-Forwarded-For: [SUPPLIED_VALUE,]CLIENT_IP,LOAD_BALANCER_IP` (see [X-Forwarded-For header](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#x-forwarded-for_header)) (requests only) The proxies also set the `X-Cloud-Trace-Context` header if it is not already present. | Yes [Configured on the backend service or backend bucket](https://docs.cloud.google.com/load-balancing/docs/https/custom-headers.md.txt) |
| Regional external Application Load Balancer | [Regional HTTP](https://docs.cloud.google.com/compute/docs/reference/rest/v1/regionTargetHttpProxies.md.txt), [Regional HTTPS](https://docs.cloud.google.com/compute/docs/reference/rest/v1/regionTargetHttpsProxies.md.txt) | - `X-Forwarded-Proto: [http | https]` (requests only) - `Via: 1.1 google` (requests and responses) - `X-Forwarded-For: [SUPPLIED_VALUE,]CLIENT_IP,LOAD_BALANCER_IP` (see [X-Forwarded-For header](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#x-forwarded-for_header)) (requests only) | Yes [Configured in the URL map](https://docs.cloud.google.com/load-balancing/docs/https/custom-headers-regional.md.txt) |
> [!NOTE]
> **Note:** In accordance with [RFC
> 2616](https://datatracker.ietf.org/doc/html/rfc2616#page-92) , the following hop-by-hop headers aren't propagated by the target proxy: `Connection`, `Keep-Alive`, `Proxy-Authenticate`, `Proxy-Authorization`, `TE`, `Trailers`, `Transfer-Encoding`, and `Upgrade`.
In addition to headers added by the target proxy, the load balancer adjusts
other HTTP headers in the following ways:
- **For the global external Application Load Balancer**, both request and response headers might be
converted to lowercase.
The only exception to this is when you use global internet NEG backends with
HTTP/1.1. For more information about how HTTP/1.1 headers are processed with
global internet NEGs, see the
[Internet NEGs overview](https://docs.cloud.google.com/load-balancing/docs/negs/internet-neg-concepts.md.txt#header-processing).
- **For the classic Application Load Balancer** , request and response headers are
converted to lowercase except when you use HTTP/1.1. With HTTP/1.1, headers
are proper-cased instead. The first letter of the header's key and every
letter following a hyphen (`-`) is capitalized to preserve compatibility
with HTTP/1.1 clients. For example, `user-agent` is changed to `User-Agent`,
and `content-encoding` is changed to `Content-Encoding`.
- Some headers are coalesced. When there are multiple instances of the same header key (for example, `Via`), the load balancer combines their values into a single comma-separated list for a single header key. Only the headers whose values can be represented as a comma-separated list are coalesced. Other headers, such as `Set-Cookie`, are never coalesced.
#### Host header
When the load balancer makes the HTTP request, the load balancer preserves the
Host header of the original request.
#### X-Forwarded-For header
The load balancer appends two IP addresses to the
`X-Forwarded-For` header, separated by a single comma, in the following order:
1. The IP address of the client that connects to the load balancer
2. The IP address of the load balancer's forwarding rule
If the incoming request does *not* include an `X-Forwarded-For` header,
the resulting header is as follows:
X-Forwarded-For: CLIENT_IP,LOAD_BALANCER_IP
If the incoming request already includes an `X-Forwarded-For` header,
the load balancer appends its values to the existing header:
X-Forwarded-For: EXISTING_VALUE,CLIENT_IP,LOAD_BALANCER_IP
> [!CAUTION]
> **Caution:** The load balancer does not verify any IP addresses that precede `CLIENT_IP,LOAD_BALANCER_IP` in this header. The preceding IP addresses might contain other characters, including spaces.
**Remove existing header values using a custom request header**
It is possible to remove existing header values by using [custom request
headers](https://docs.cloud.google.com/load-balancing/docs/https/custom-headers#working-with-request) on the
backend service. The following example uses the `--custom-request-header` flag
to recreate the `X-Forwarded-For` header by using the variables
`client_ip_address` and `server_ip_address`. This configuration replaces the
incoming `X-Forwarded-For` header with only the client and the load balancer IP
address.
--custom-request-header=x-forwarded-for:{client_ip_address},{server_ip_address}
> [!NOTE]
> **Note:** This approach discards any existing `X-Forwarded-For` values from the request. Because Google Cloud doesn't log header values, these values can't be recovered. If you need to retain the original header content, you can store it in a separate custom header.
**How backend reverse proxy software might modify the `X-Forwarded-For` header**
If your load balancer's backends run HTTP reverse proxy software,
the software might append one or both of the following IP addresses to the
end of the `X-Forwarded-For` header:
1. The IP address of the GFE that connected to the backend.
The GFE proxy ranges are listed in the
[Firewall rules](https://docs.cloud.google.com/load-balancing/docs/firewall-rules.md.txt) document.
2. The IP address of the backend system itself.
As a result, an upstream system might see an `X-Forwarded-For` header
structured as follows:
X-Forwarded-For: EXISTING_VALUE,CLIENT_IP,LOAD_BALANCER_IP,GFE_IP,BACKEND_IP
For example:
X-Forwarded-For: 192.0.2.50,203.0.113.10,198.51.100.5,35.191.0.20,10.128.0.2
In this example, only the two addresses that the load balancer appends
(`203.0.113.10,198.51.100.5`, corresponding to
`CLIENT_IP,LOAD_BALANCER_IP`) can be trusted.
Any values before them come from the client and aren't verified by the load
balancer.
#### Cloud Trace support
Trace is not supported with Application Load Balancers. The global
and classic Application Load Balancers add the `X-Cloud-Trace-Context` header if it is not
present. The regional external Application Load Balancer doesn't add this header. If the
`X-Cloud-Trace-Context` header is already present, it passes through the load
balancers unmodified. However, no traces or spans are exported by the load
balancer.
### URL maps
[URL maps](https://docs.cloud.google.com/load-balancing/docs/url-map-concepts.md.txt) define matching patterns for
URL-based routing of requests to the appropriate backend services. The URL map
allows you to divide your traffic by examining the URL components to
send requests to different sets of backends. A default service is defined to
handle any requests that don't match a specified host rule or path matching
rule.
In some situations, such as the [multi-region load
balancing example](https://docs.cloud.google.com/load-balancing/docs/https/setting-up-https), you might not
define any URL rules and rely only on the default service.
URL maps support several advanced traffic management features such as
header-based traffic steering, weight-based traffic splitting, and request
mirroring. For more information, see the following:
- [Traffic management overview for
global external Application Load Balancer](https://docs.cloud.google.com/load-balancing/docs/https/traffic-management-global)
- [Traffic management overview for
regional external Application Load Balancer](https://docs.cloud.google.com/load-balancing/docs/https/traffic-management-regional)
The following table specifies the type of URL map required by external Application Load Balancers in
each mode.
| Load balancer mode | URL map type |
|---|---|
| Global external Application Load Balancer | [Global](https://docs.cloud.google.com/compute/docs/reference/rest/v1/urlMaps.md.txt) |
| Classic Application Load Balancer | [Global](https://docs.cloud.google.com/compute/docs/reference/rest/v1/urlMaps.md.txt) (with only a [subset of the features supported](https://docs.cloud.google.com/load-balancing/docs/features.md.txt#routing-traffic-management)) |
| Regional external Application Load Balancer | [Regional](https://docs.cloud.google.com/compute/docs/reference/rest/v1/regionUrlMaps.md.txt) |
### SSL certificates
External Application Load Balancers using target HTTPS proxies require private keys and
SSL certificates as part of the load balancer configuration.
- Google Cloud provides two *configuration methods* for assigning private
keys and SSL certificates to target HTTPS proxies: Compute Engine SSL
certificates and Certificate Manager. For a description of each
configuration, see [Certificate configuration
methods](https://docs.cloud.google.com/load-balancing/docs/ssl-certificates#config-tech) in the SSL
certificates overview.
- Google Cloud provides two *certificate types* : Self-managed and
Google-managed. For a description of each type, see [Certificate
types](https://docs.cloud.google.com/load-balancing/docs/ssl-certificates#certificate-types) in the SSL
certificates overview.
The type of external Application Load Balancer (global, regional, or classic) determines which
configuration methods and certificate types are supported. For details, see
[Certificates and Google Cloud load
balancers](https://docs.cloud.google.com/load-balancing/docs/ssl-certificates#certificate-summary) in the
SSL certificates overview.
### SSL policies
[SSL policies](https://docs.cloud.google.com/load-balancing/docs/ssl-policies-concepts.md.txt) specify the set of
SSL features that Google Cloud load balancers use when negotiating SSL
with clients.
By default, HTTPS Load Balancing uses a set of SSL features that provides good
security and wide compatibility. Some applications require more control over
which SSL versions and ciphers are used for their HTTPS or SSL connections. You
can define an SSL policy to specify the set of SSL features that your load
balancer uses when negotiating SSL with clients. In addition, you can apply that
SSL policy to your target HTTPS proxy.
The following table specifies the SSL policy support for load balancers in each
mode.
| Load balancer mode | SSL policies supported |
|---|---|
| Global external Application Load Balancer | Yes |
| Classic Application Load Balancer | Yes |
| Regional external Application Load Balancer | Yes |
### Backend services
A backend service provides configuration information to the load balancer so
that it can direct requests to its backends---for example,
Compute Engine instance groups or network endpoint groups (NEGs). For
more information about backend services, see [Backend services
overview](https://docs.cloud.google.com/load-balancing/docs/backend-service).
For an example showing how to set up a load balancer with a backend service and
a Compute Engine backend, see [Setting up an external Application Load Balancer with a
Compute Engine backend](https://docs.cloud.google.com/load-balancing/docs/https/ext-https-lb-simple).
#### Backend service scope
The following table indicates which backend service resource and scope is used
by external Application Load Balancers:
| Load balancer mode | Backend service resource |
|---|---|
| Global external Application Load Balancer | [`backendServices` (global)](https://docs.cloud.google.com/compute/docs/reference/rest/v1/backendServices.md.txt) |
| Classic Application Load Balancer | [`backendServices` (global)](https://docs.cloud.google.com/compute/docs/reference/rest/v1/backendServices.md.txt) |
| Regional external Application Load Balancer | [`regionBackendServices` (regional)](https://docs.cloud.google.com/compute/docs/reference/rest/v1/regionBackendServices.md.txt) |
#### Protocol to the backends
Backend services for Application Load Balancers must use one of the following
protocols to send requests to backends:
- HTTP, which uses HTTP/1.1 and no TLS
- HTTPS, which uses HTTP/1.1 and TLS
- HTTP/2, which uses HTTP/2 and TLS (HTTP/2 without encryption isn't supported.)
- H2C, which uses HTTP/2 over TCP. TLS isn't required. H2C isn't supported for classic Application Load Balancers.
The load balancer only uses the backend service protocol that you specify to
communicate with its backends. The load balancer doesn't fall back to a
different protocol if it is unable to communicate with backends using the
specified backend service protocol.
The backend service protocol doesn't need to match the protocol used by clients
to communicate with the load balancer. For example, clients can send requests
to the load balancer using HTTP/2, but the load balancer can communicate with
backends using HTTP/1.1 (HTTP or HTTPS).
### Backend buckets
Backend buckets direct incoming traffic to Cloud Storage buckets. For an
example showing how to add a bucket to an external Application Load Balancer, see [Setting up a
load balancer with backend
buckets](https://docs.cloud.google.com/load-balancing/docs/https/ext-load-balancer-backend-buckets). For
general information about Cloud Storage, see [What is
Cloud Storage?](https://docs.cloud.google.com/storage/docs/introduction)
### Backends
The following table specifies the backends and related features supported by
external Application Load Balancers in each mode.
| Load balancer mode | Supported backends on a backend service^1^ |||||| Supports backend buckets | Supports Cloud Armor | Supports Cloud CDN^2^ | Supports IAP^2^ | Supports Service Extensions |
|---|---|---|---|---|---|---|---|---|---|---|---|
| | **Instance groups** ^3^ | **Zonal NEGs** ^4^ | **Internet NEGs** | **Serverless NEGs** | **Hybrid NEGs** | **Private Service Connect NEGs** | | | | | |
| Global external Application Load Balancer | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| Classic Application Load Balancer | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes Premium Tier | Yes | No |
| Regional external Application Load Balancer | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes |
^1^Backends on a backend service must be the same type: all instance
groups or all the same type of NEG. An exception to this rule is that both
`GCE_VM_IP_PORT` zonal NEGs and hybrid NEGs can be used on the same
backend service to support a
[hybrid architecture](https://docs.cloud.google.com/load-balancing/docs/negs/hybrid-neg-concepts.md.txt#hybrid_architecture).
^2^ IAP and Cloud CDN are incompatible with
each other. They can't both be enabled on the same backend service.
^3^ Combinations of zonal unmanaged, zonal managed, and regional
managed instance groups are supported on the same backend service. When using
autoscaling for a managed instance group that's a backend for two or more
backend services, configure the instance group's autoscaling policy to use
[multiple signals](https://docs.cloud.google.com/compute/docs/autoscaler/multiple-signals.md.txt).
^4^ Zonal NEGs must use `GCE_VM_IP_PORT` endpoints.
#### Backends and VPC networks
The restrictions on where backends can be located depend on the type of
load balancer.
**For global external Application Load Balancer backends**, the following applies:
- Backend instances (for instance group
backends) and backend endpoints (for NEG backends) can be located in any
VPC network *within the same organization*. The different
VPC networks don't need to be connected using
VPC Network Peering because GFEs communicate directly
with backends in their respective VPC networks.
- Cloud Storage buckets aren't associated with a VPC
network. They can be located in any project within the same
organization.
Global external Application Load Balancers also support [Shared VPC
environments](https://docs.cloud.google.com/load-balancing/docs/https#shared-vpc) where you can share VPC networks and
their associated resources across projects. However, for
global external Application Load Balancers, you don't need to configure Shared VPC to be
able to reference backend services or backend buckets from other projects in
your organization.
**For classic Application Load Balancer backends**, the following applies:
- All backend instances from instance group backends and all backend endpoints
from NEG backends must be located *in the
same project*. However, an instance group backend or a NEG can use a different
VPC network in that project. The different VPC
networks don't need to be connected using VPC Network Peering because GFEs
communicate directly with backends in their respective VPC
networks.
- Cloud Storage buckets aren't associated with a VPC
network. However, they must be located in the same project as the load
balancer.
**For regional external Application Load Balancer backends**, the following applies:
- For instance groups, zonal NEGs, and hybrid connectivity NEGs, all backends
must be located in the same project and region as the backend service.
However, a load balancer can reference a backend that uses a different
VPC network in the same project as the backend service.
Connectivity between the load balancer's
VPC network and the backend VPC network
can be configured using either VPC Network Peering, Cloud VPN
tunnels, Cloud Interconnect VLAN attachments, or a Network Connectivity Center
framework.
**Backend network definition**
- For zonal NEGs and hybrid NEGs, you explicitly specify the VPC network when you create the NEG.
- For managed instance groups, the VPC network is defined in the instance template.
- For unmanaged instance groups, the instance group's VPC network is set to match the VPC network of the `nic0` interface for the first VM added to the instance group.
**Backend network requirements**
Your backend's network must satisfy *one of the following* network
requirements:
- The backend's VPC network must exactly match the
forwarding rule's VPC network.
- The backend's VPC network must be connected to the
forwarding rule's VPC network using
VPC Network Peering. You must configure subnet route exchanges to
allow communication between the proxy-only subnet in the forwarding rule's
VPC network and the subnets used by the backend instances
or endpoints.
**Backend IPv6 subnet requirements**
The IP version used for the frontend connection is independent of the backend
connection. If the proxy-only subnet is configured as dual-stack (IPV4_IPV6),
the proxy-only subnet can communicate with backends using either IPv4 or
IPv6. If the proxy-only subnet is single-stack (IPV4_ONLY), it can only
communicate with backends using IPv4.
If your backend instances are handling IPv6 traffic, the backend subnet can
be configured with a stack type of `IPV4_ONLY` or
`IPV4_IPV6` (dual-stack). If the backend subnet's stack type
includes IPv6, you must explicitly set the subnet's ipv6-access-type to
`EXTERNAL`.
- Both the backend's VPC network and the forwarding rule's VPC network must be [VPC
spokes](https://docs.cloud.google.com/network-connectivity/docs/network-connectivity-center/concepts/vpc-spokes-overview) attached to the same [NCC
hub](https://docs.cloud.google.com/network-connectivity/docs/network-connectivity-center/concepts/overview). Import and export filters must allow communication between the proxy-only subnet in the forwarding rule's VPC network and the subnets used by backend instances or endpoints.
- For all other backend types, all backends must be located in the same
VPC network and region.
Regional external Application Load Balancers also support Shared VPC
environments where you can share VPC networks and their
associated resources across projects. If you want the
regional external Application Load Balancer's backend service and backends to be in a different
project from the forwarding rule, you need to configure the load balancer in a
[Shared VPC environment with cross-project service
referencing](https://docs.cloud.google.com/load-balancing/docs/https#shared-vpc).
#### Backends and network interfaces
If you use instance group backends, packets are always delivered to `nic0`. If
you want to send packets to non-`nic0` interfaces (either [vNICs or
Dynamic Network Interfaces](https://docs.cloud.google.com/vpc/docs/multiple-interfaces-concepts#nic-types)), use
NEG backends instead.
If you use zonal NEG backends, packets are sent to whatever network interface is
represented by the endpoint in the NEG. The NEG endpoints must be in the same
VPC network as the NEG's explicitly defined VPC
network.
### Health checks
Each backend service specifies a health check that periodically monitors the
backends' readiness to receive a connection from the load balancer. This reduces
the risk that requests might be sent to backends that can't service the request.
Health checks don't check if the application itself is working.
For the health check probes, you must create an ingress allow [firewall
rule](https://docs.cloud.google.com/load-balancing/docs/https#firewall-rules) that allows health check probes to reach your backend
instances. Typically, health check probes originate from Google's centralized
health checking mechanism.
Regional external Application Load Balancers that use hybrid NEG backends are an exception to
this rule because their health checks originate from the proxy-only subnet
instead. For more information, see the
[Hybrid NEGs overview](https://docs.cloud.google.com/load-balancing/docs/negs/hybrid-neg-concepts.md.txt#envoy-health-checks).
#### Health check protocol
Although it is not required and not always possible, it is a best practice to
use a health check whose protocol matches the [protocol of the backend
service](https://docs.cloud.google.com/load-balancing/docs/features#protocol-to-backends).
For example, an HTTP/2 health check most accurately tests HTTP/2 connectivity to
backends. In contrast, regional external Application Load Balancers that use hybrid NEG backends
don't support gRPC health checks. For the list of supported health check
protocols, see [Load balancing
features](https://docs.cloud.google.com/load-balancing/docs/features#health-checks).
The following table specifies the scope of health checks supported by
external Application Load Balancers in each mode.
| Load balancer mode | Health check type |
|---|---|
| Global external Application Load Balancer | [Global](https://docs.cloud.google.com/compute/docs/reference/rest/v1/healthChecks.md.txt) |
| Classic Application Load Balancer | [Global](https://docs.cloud.google.com/compute/docs/reference/rest/v1/healthChecks.md.txt) |
| Regional external Application Load Balancer | [Regional](https://docs.cloud.google.com/compute/docs/reference/rest/v1/regionHealthChecks.md.txt) |
For more information about health checks, see the following:
- [Health checks overview](https://docs.cloud.google.com/load-balancing/docs/health-check-concepts.md.txt)
- [Creating health checks](https://docs.cloud.google.com/load-balancing/docs/health-checks.md.txt)
### Firewall rules
The load balancer requires the following firewall rules:
- For the global external Application Load Balancers, an ingress allow rule to permit traffic from Google Front Ends (GFEs) to reach your backends. For the regional external Application Load Balancer, an ingress allow rule to permit traffic from the [proxy-only subnet](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#proxy-only-subnet) to reach your backends.
- An ingress allow rule to permit traffic from the health check probes ranges. For more information about health check probes and why it's necessary to allow traffic from them, see [Probe IP ranges and firewall
rules](https://docs.cloud.google.com/load-balancing/docs/health-check-concepts#ip-ranges).
Firewall rules are implemented at the VM instance level, not on GFE proxies. You
cannot use Google Cloud firewall rules to prevent traffic from reaching
the load balancer. For the global external Application Load Balancer and the classic Application Load Balancer, you can use
[Google Cloud Armor](https://docs.cloud.google.com/armor/docs.md.txt) to achieve this.
The ports for these firewall rules must be configured as follows:
- Allow traffic to the destination port for each backend service's health check.
- For instance group backends: Determine the ports to be configured by the
mapping between the backend service's [named
port](https://docs.cloud.google.com/load-balancing/docs/backend-service#named_ports) and
the port numbers associated with that named port on each instance group. The
port numbers can vary among instance groups assigned to the same backend
service.
- For `GCE_VM_IP_PORT` NEG backends: Allow traffic to the port numbers of the
endpoints.
The following table summarizes the required source IP address ranges for the
firewall rules:
| Load balancer mode | Health check source ranges | Request source ranges |
|---|---|---|
| Global external Application Load Balancer | For IPv4 health checks to the backends: - `35.191.0.0/16` For IPv6 health checks to the backends: - `2600:2d00:1:b029::/64` | The source of GFE traffic depends on the backend type: - Instance groups and zonal NEGs (`GCE_VM_IP_PORT`): - `130.211.0.0/22` - `35.191.0.0/16` For IPv6 traffic to the backends: - `2600:2d00:1:1::/64` - Hybrid connectivity NEGs (`NON_GCP_PRIVATE_IP_PORT`): - `130.211.0.0/22` - `35.191.0.0/16` - Internet NEGs (`INTERNET_FQDN_PORT` and `INTERNET_IP_PORT`): - IP address ranges listed in the `_cloud-eoips.googleusercontent.com` DNS TXT record. You can extract the source IP addresses for global internet NEG backends using the following example command on a Linux system: `dig TXT _cloud-eoips.googleusercontent.com | grep -Eo 'ip4:[^ ]+' | cut -d':' -f2` - Serverless NEGs and backend buckets: Google's production network handles packet routing |
| Classic Application Load Balancer | For IPv4 health checks to the backends: - `35.191.0.0/16` | The source of GFE traffic depends on the backend type: - Instance groups, zonal NEGs (`GCE_VM_IP_PORT`), and hybrid connectivity NEGs (`NON_GCP_PRIVATE_IP_PORT`): - `130.211.0.0/22` - `35.191.0.0/16` - Internet NEGs (`INTERNET_FQDN_PORT` and `INTERNET_IP_PORT`): - IP address ranges listed in the `_cloud-eoips.googleusercontent.com` DNS TXT record. You can extract the source IP addresses for global internet NEG backends using the following example command on a Linux system: `dig TXT _cloud-eoips.googleusercontent.com | grep -Eo 'ip4:[^ ]+' | cut -d':' -f2` - Serverless NEGs and backend buckets: Google's production network handles packet routing. |
| Regional external Application Load Balancer | For IPv4 health checks to the backends: - `35.191.0.0/16` For IPv6 health checks to the backends: - `2600:2d00:1:b029::/64` Allowing traffic from Google's health check probe ranges isn't required for hybrid NEGs. However, if you're using a combination of hybrid and zonal NEGs in a single backend service, you need to allow traffic from the [Google health check probe ranges](https://docs.cloud.google.com/load-balancing/docs/health-check-concepts.md.txt#ip-ranges) for the zonal NEGs. | The [proxy-only subnet](https://docs.cloud.google.com/load-balancing/docs/proxy-only-subnets.md.txt#proxy_only_subnet_create) that you configure. If you are using a dual-stack proxy-only subnet to proxy IPv6 traffic, your firewall rule should explicitly set the IPv6 source range to ensure traffic can successfully reach the backends. For example, using `--source-ranges ::/0` or the proxy-only subnet's specific IPv6 range. |
> [!IMPORTANT]
> **Important:** Make sure to allow packets from the full health check ranges. If your firewall rule allows packets from only a subset of the ranges, you might see health check failures because the load balancer can't communicate with your backends. This causes HTTP `502` status code responses.
## Global Front End
> [!NOTE]
> **Note:** A Global Front End is *different* from a Google Front End (GFE). The GFE is a reverse proxy that terminates TCP connections, whereas a Global Front End is a solution that helps deliver, scale, and secure your internet-facing applications. Global Front End Enterprise consolidates the pricing of different Google Cloud networking products into one bundled pricing tier.
Global Front End, available in [Preview](https://docs.cloud.google.com/products#product-launch-stages),
combines global external Application Load Balancers, Google Cloud Armor,
Cloud CDN, and Service Extensions into one solution
to help deliver, scale, and secure your
internet-facing applications. You can enable the
Global Front End Enterprise billing tier in your Google Cloud
project. With Global Front End Enterprise, pricing for
the aforementioned products is consolidated under
a [single pricing plan](https://docs.cloud.google.com/vpc/network-pricing#global-front-end-enterprise).
For more information on
how to enable the Global Front End Enterprise
billing tier in your project,
see [Enable and manage Global Front End Enterprise in your
Google Cloud project](https://docs.cloud.google.com/docs/networking/cross-cloud-network/global-front-end/gfee-manage).
When you enable Global Front End Enterprise in a project, all
global external Application Load Balancers in that project gain access to
[specific features of Cloud Armor Enterprise](https://docs.cloud.google.com/armor/docs/armor-enterprise-overview.md.txt#global-front-end-enterprise).
In a [Global Front End architecture](https://docs.cloud.google.com/docs/networking/cross-cloud-network/global-front-end/gfee-overview.md.txt#services),
a global external Application Load Balancer distributes HTTP and HTTPS traffic across
an application's backend instances in multiple
regions. Cloud CDN accelerates
internet-facing applications by using Google's edge points of presence (PoPs)
and works with the global external Application Load Balancer to deliver content to users.
Google Cloud Armor security policies protect the backend instances.
These policies provide Layer 7 filtering by inspecting
incoming requests for common web attacks or other Layer 7 attributes,
blocking traffic before it reaches the load-balanced backend services.
Service Extensions lets you customize the
data processing of your load balancer to suit your specific application or
business requirements.
> [!NOTE]
> **Note:** If a global external Application Load Balancer routes traffic to backends in other projects, Global Front End Enterprise pricing applies only to the project where the load balancer frontend components reside. Usage and charges in projects that contain the backend components follow individual product pricing.
## GKE support
GKE uses external Application Load Balancers in the following ways:
- External Gateways created using the [GKE Gateway
controller](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/gateway-api) can use any mode of
an external Application Load Balancer. You control the load balancer's mode by choosing a
[GatewayClass](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/gateway-api.md.txt#gatewayclass). The
GKE Gateway controller always uses `GCE_VM_IP_PORT` zonal NEG
backends.
When Cloud CDN is enabled for backends managed by GKE
Gateway, caching is configured using the GKE `GCPHTTPFilter`
filters that are attached to `HTTPRoute` resources. For information about how to
configure caching, see [Configure Cloud CDN for Gateway](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/configure-cdn-for-gateway.md.txt).
- External Ingresses created using the [GKE Ingress
controller](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/ingress) are always classic Application Load Balancers. The GKE Ingress controller prefers to use `GCE_VM_IP_PORT` zonal NEG backends, though instance group backends are also supported.
- You can use `GCE_VM_IP_PORT` zonal NEG created and managed by GKE Services as backends for any Application Load Balancer or Proxy Network Load Balancer. For more information, see [Container-native load
balancing through standalone zonal
NEGs](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/standalone-neg).
## Shared VPC architecture
External Application Load Balancers support networks that use Shared VPC.
Shared VPC lets organizations connect resources from multiple projects
to a common VPC network so that they can communicate with each
other securely and efficiently by using internal IP addresses from that network.
If you're not already familiar with Shared VPC, read the [Shared VPC overview](https://docs.cloud.google.com/vpc/docs/shared-vpc.md.txt).
There are many ways to configure an external Application Load Balancer within a Shared VPC
network. Regardless of type of deployment, all the components of the load
balancer must be in the same organization.
| Load balancer | Frontend components | Backend components |
|---|---|---|
| Global external Application Load Balancer | If you're using a [Shared VPC network](https://docs.cloud.google.com/vpc/docs/shared-vpc.md.txt#shared_vpc_networks) for your backends, create the required network in the Shared VPC host project. The global external IP address, the forwarding rule, the target HTTP(S) proxy, and the associated URL map must be defined in the same project. This project can be a host project or a service project. | You can do one of the following: - Create backend services, backend buckets, and [backends](https://docs.cloud.google.com/load-balancing/docs/features.md.txt#backends) (instance groups, serverless NEGs, or any other supported backend types) in the [same service project](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#service-project) as the frontend components. - Create backend services, backend buckets, and backends (instance groups, serverless NEGs, or any other supported backend types) in service projects. A single URL map can reference backend services across different projects. This type of deployment is known as [cross-project service referencing](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#cross-project). Each backend service must be defined in the same project as the backends it references. Health checks associated with backend services must also be defined in the same project as the backend service. Backends can be a part of either a Shared VPC network from the host project or a standalone VPC network---that is, an unshared VPC network in the service project. |
| Classic Application Load Balancer | The global external IP address, the forwarding rule, the target HTTP(S) proxy, and the associated URL map must be defined in the same host or service project as the backends. | A global backend service must be defined in the same host or service project as the [backends](https://docs.cloud.google.com/load-balancing/docs/features.md.txt#backends) (instance groups or NEGs). Health checks associated with backend services must be defined in the same project as the backend service as well. |
| Regional external Application Load Balancer | Create the required network and proxy-only subnet in the Shared VPC host project. The regional external IP address, the forwarding rule, the target HTTP(S) proxy, and the associated URL map must be defined in the same project. This project can be the host project or a service project. | You can do one of the following: - Create backend services and [backends](https://docs.cloud.google.com/load-balancing/docs/features.md.txt#backends) (instance groups, serverless NEGs, or any other supported backend types) in the [same service project](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#service-project) as the frontend components. - Create backend services and backends (instance groups, serverless NEGs, or any other supported backend types) in as many service projects as required. A single URL map can reference backend services across different projects. This type of deployment is known as [cross-project service referencing](https://docs.cloud.google.com/load-balancing/docs/https.md.txt#cross-project). Each backend service must be defined in the same project as the backends it references. Health checks associated with backend services must be defined in the same project as the backend service as well. |
While you can create all the load balancing components and backends in the
Shared VPC host project, this type of deployment doesn't separate
network administration and service development responsibilities.
#### All load balancer components and backends in a service project
The following architecture diagram shows a standard Shared VPC
deployment where all load balancer components and backends are in a service
project. This deployment type is supported by all Application Load Balancers.
The load balancer components and backends must use the same VPC
network.
[](https://docs.cloud.google.com/static/load-balancing/images/reg-ext-https-lb-l7-shared-vpc-example.svg) Regional external Application Load Balancer on Shared VPC network (click to enlarge).
#### Serverless backends in a Shared VPC environment
For a load balancer that is using a serverless NEG backend, the backend
Cloud Run or Cloud Run functions service must be in the
same project as the serverless NEG.
Additionally, for the regional external Application Load Balancer that supports cross-project
service referencing, the backend service, serverless NEG, and the
Cloud Run service must always be in the same service project.
## Cross-project service referencing
*Cross-project service referencing* is a deployment model where the load
balancer's frontend and URL map are in one project and the load balancer's
backend service and backends are in a different project.
Cross-project service referencing lets organizations configure one central
load balancer and route traffic to hundreds of services distributed across
multiple different projects. You can centrally manage all traffic routing rules
and policies in one URL map. You can also associate the load balancer with a
single set of hostnames and SSL certificates. You can therefore optimize the
number of load balancers needed to deploy your application, and lower
manageability, operational costs, and quota requirements.
By having different projects for each of your functional teams, you can also
achieve separation of roles within your organization. Service owners can focus
on building services in service projects, while network teams can provision and
maintain load balancers in another project, and both can be connected by using
cross-project service referencing.
Service owners can maintain autonomy over the exposure of their services and
control which users can access their services by using the load balancer. This is
achieved by a special IAM role called the
[Compute Load Balancer Services User role](https://docs.cloud.google.com/compute/docs/access/iam.md.txt#compute.loadBalancerServiceUser)
(`roles/compute.loadBalancerServiceUser`).
Cross-project service referencing support differs based on the type of
load balancer:
- **For global external Application Load Balancers** : your load balancer's frontend and URL map can
reference backend services or backend buckets from any project within the same
organization. No VPC network restrictions apply. While you can
use a Shared VPC environment to configure a cross-project deployment
as shown in this
[example](https://docs.cloud.google.com/load-balancing/docs/https/set-up-global-ext-https-shared-vpc.md.txt), this
isn't a requirement.
- **For regional external Application Load Balancers**: you must create the load balancer in a
Shared VPC environment. The load balancer's frontend and URL map must
be in a host or service project, and the load balancer's backend services and
backends can be distributed across host or service projects in the same
Shared VPC environment.
To learn how to configure Shared VPC for a global external Application Load Balancer---with and without cross-project service referencing---see [Set up a global external Application Load Balancer with
Shared VPC](https://docs.cloud.google.com/load-balancing/docs/https/set-up-global-ext-https-shared-vpc).
To learn how to configure Shared VPC for a
regional external Application Load Balancer---with and without cross-project service
referencing---see [Set up a regional external Application Load Balancer with
Shared VPC](https://docs.cloud.google.com/load-balancing/docs/https/setting-up-reg-ext-shared-vpc).
#### Usage notes for cross-project service referencing
- Cross-project service referencing can be used with instance groups,
serverless NEGs, and most other supported backend types. However, the
following limitations apply:
- With global external Application Load Balancers, you can't reference a cross-project
backend service if the backend service has serverless NEG backends
with App Engine.
- With regional external Application Load Balancers, you can't reference a cross-project backend service if the backend service has regional internet NEG backends.
- Cross-project service referencing is *not supported* for the classic Application Load Balancer.
- Google Cloud doesn't differentiate between resources (for example, backend services) using the same name across multiple projects. Therefore, when you are using cross-project service referencing, we recommend that you use unique backend service names across projects within your organization.
- If you see an error such as "Cross-project references for this resource are not allowed", make sure that you have the permission to use the resource. An administrator of the project that owns the resource must grant you the [Compute
Load Balancer Services User role](https://docs.cloud.google.com/compute/docs/access/iam#compute.loadBalancerServiceUser) (`roles/compute.loadBalancerServiceUser`). This role can be granted at the project level or at the resource level. For an example, see [Grant
permissions to the Compute Load Balancer Admin to use the backend
service or backend bucket](https://docs.cloud.google.com/load-balancing/docs/https/set-up-global-ext-https-shared-vpc#grant-bs-user).
#### Example 1: Load balancer frontend and backend in different service projects
Here is an example of a Shared VPC deployment where the load balancer's
frontend and URL map are created in service project A and the URL map references
a backend service in service project B.
In this case, Network Admins or Load Balancer Admins in service project A
require access to backend services in service project B. Service project B
admins grant the Compute Load Balancer Services User role
(`roles/compute.loadBalancerServiceUser`) to Load Balancer Admins in
service project A who want to reference the backend
service in service project B.
[](https://docs.cloud.google.com/static/load-balancing/images/reg-ext-https-lb-l7-shared-vpc-service-lb.svg) Load balancer frontend and backend in different service projects (click to enlarge).
#### Example 2: Load balancer frontend in the host project and backends in service projects
Here is an example of a Shared VPC deployment where the load balancer's
frontend and URL map are created in the host project and the backend services
(and backends) are created in service projects.
In this case, Network Admins or Load Balancer Admins in the host project
require access to backend services in the service project. Service project
admins grant the Compute Load Balancer Services User role
(`roles/compute.loadBalancerServiceUser`) to
Load Balancer Admins in the host project A who want to reference the backend
service in the service project.
[](https://docs.cloud.google.com/static/load-balancing/images/reg-ext-https-lb-l7-shared-vpc-host-lb.svg) Load balancer frontend and URL map in host project (click to enlarge).
#### Example 3: Load balancer frontend and backends in different projects
Here is an example of a deployment where the global external Application Load Balancer's frontend
and URL map are created in a different project from the load balancer's backend
service and backends. This type of deployment doesn't use Shared VPC
and is supported only for global external Application Load Balancers.
[](https://docs.cloud.google.com/static/load-balancing/images/global-ext-https-lb-cross-proj.svg) Load balancer frontend and backends in different projects (click to enlarge).
To learn more about this setup, see
[Set up cross-project service referencing with a backend service and a backend bucket](https://docs.cloud.google.com/load-balancing/docs/https/setup-cross-project-backend-service-backend-bucket.md.txt).
## High availability and failover
High availability and failover in external Application Load Balancers can be configured at the load
balancer level. This is handled by deploying regional external Application Load Balancers in the
regions where you require backup.
The following table describes the high availability and failover methods:
| Load balancer mode | Failover methods |
|---|---|
| Global external Application Load Balancer Classic Application Load Balancer | Use one of the following methods to protect against outages: - You can configure an [active-passive](https://docs.cloud.google.com/load-balancing/docs/https/global-applb-failover-overview.md.txt#active-passive) global-to-regional failover configuration in which traffic fails over from a primary global external Application Load Balancer or classic Application Load Balancer to backup regional external Application Load Balancers. You use health checks to detect outages and a Cloud DNS [failover routing policy](https://docs.cloud.google.com/dns/docs/routing-policies-overview.md.txt#failover-policy) to route traffic when failover is triggered. - For global external Application Load Balancers, you can configure an [active-active](https://docs.cloud.google.com/load-balancing/docs/https/global-applb-failover-overview.md.txt#active-active) global-to-regional bypass configuration where the global external Application Load Balancer acts as an edge frontend that forwards requests to regional external Application Load Balancers using internet NEGs with FQDNs. If the global load balancer experiences an outage, a Cloud DNS [geolocation routing policy](https://docs.cloud.google.com/dns/docs/routing-policies-overview.md.txt#geolocation-policy) bypasses the global layer and routes client traffic directly to the regional load balancers. For more information, see [Failover for global external Application Load Balancers](https://docs.cloud.google.com/load-balancing/docs/https/global-applb-failover-overview.md.txt). |
| Regional external Application Load Balancer | Use one of the following methods to ensure a highly available deployment: - You can configure an [active-passive](https://docs.cloud.google.com/load-balancing/docs/https/reg-applb-ha-overview.md.txt#strategies) failover configuration in which traffic fails over to a backup regional external Application Load Balancer. You use health checks to detect outages and a Cloud DNS [failover routing policy](https://docs.cloud.google.com/dns/docs/routing-policies-overview.md.txt#failover-policy) to route traffic when failover is triggered by failing health checks. - You can configure an [active-active](https://docs.cloud.google.com/load-balancing/docs/https/reg-applb-ha-overview.md.txt#strategies) high availability configuration where you deploy multiple individual regional external Application Load Balancers in regions that you determine best support traffic for your application. You use a Cloud DNS [geolocation routing policy](https://docs.cloud.google.com/dns/docs/routing-policies-overview.md.txt#geolocation-policy) to route traffic to the appropriate region based on the origin of the client request. You can also use health checks to detect outages and route traffic only to the healthy load balancers. For more information, see [High availability for regional external Application Load Balancers](https://docs.cloud.google.com/load-balancing/docs/https/reg-applb-ha-overview.md.txt). |
## HTTP/2 support
HTTP/2 is a major revision of the HTTP/1 protocol. There are 2 modes of HTTP/2
support:
- HTTP/2 over TLS
- Cleartext HTTP/2 over TCP
### HTTP/2 over TLS
HTTP/2 over TLS is supported for connections between clients and the
external Application Load Balancer, and for connections between the load balancer and its backends.
The load balancer automatically negotiates HTTP/2 with clients as part of the
TLS handshake by using the ALPN TLS extension. Even if a load balancer is
configured to use HTTPS, modern clients default to HTTP/2. This is controlled
*on the client*, not on the load balancer.
If a client doesn't support HTTP/2 and the load balancer is configured to use
HTTP/2 between the load balancer and the backend instances, the load balancer
might still negotiate an HTTPS connection or accept unsecured HTTP requests.
Those HTTPS or HTTP requests are then transformed by the load balancer to proxy
the requests over HTTP/2 to the backend instances.
To use HTTP/2 over TLS, you must enable TLS on your backends and set the
[backend service protocol to
`HTTP2`](https://docs.cloud.google.com/sdk/gcloud/reference/compute/backend-services/create#--protocol). For
more information, see [Encryption from the load balancer to the
backends](https://docs.cloud.google.com/load-balancing/docs/ssl-certificates/encryption-to-the-backends).
#### HTTP/2 max concurrent streams
The HTTP/2
[`SETTINGS_MAX_CONCURRENT_STREAMS`](https://tools.ietf.org/html/rfc7540#section-6.5.2)
setting describes the maximum number of streams that an endpoint accepts,
initiated by the peer. The value advertised by an HTTP/2 client to a
Google Cloud load balancer is effectively meaningless because the load
balancer doesn't initiate streams to the client.
In cases where the load balancer uses HTTP/2 to communicate with a server that
is running on a VM, the load balancer respects the
`SETTINGS_MAX_CONCURRENT_STREAMS` value advertised by the server, up to a
maximum value of `100`. In the request direction (Google Cloud load
balancer → gRPC server), the load balancer uses the initial `SETTINGS` frame
from the gRPC server to determine how many streams per connection can be in use
simultaneously. If the server advertises a value higher than `100`, the load
balancer uses 100 as the maximum number of concurrent streams. If a value of
zero is advertised, the load balancer can't forward requests to the server, and
this might result in errors.
#### HTTP/2 dynamic header table size
HTTP/2 significantly improves upon HTTP/1.1 with features like multiplexing
and HPACK header compression. HPACK uses a dynamic table that enhances header
compression, making everything faster. To understand the impact of dynamic
header table size changes in HTTP/2, how this feature can improve performance
and how a specific bug in various HTTP client libraries could cause issues
in HPACK header compression, refer to the [community
article](https://discuss.google.dev/t/understanding-the-impact-of-dynamic-header-table-size-changes-in-http-2/256423).
#### HTTP/2 limitations
- HTTP/2 between the load balancer and the instance can require significantly more TCP connections to the instance than HTTP or HTTPS. Connection pooling, an optimization that reduces the number of these connections with HTTP or HTTPS, isn't available with HTTP/2. As a result, you might see high backend latencies because backend connections are made more frequently.
- HTTP/2 between the load balancer and the backend doesn't support server push.
- The gRPC error rate and request volume aren't visible in the Google Cloud API or the Google Cloud console. If the gRPC endpoint returns an error, the load balancer logs and the monitoring data report the `200 OK` HTTP status code.
### HTTP/2 over cleartext TCP
HTTP/2 over cleartext TCP, represented by the string "h2c" per [RFC 7540](https://datatracker.ietf.org/doc/html/rfc7540#section-11.1),
lets you use HTTP/2 without TLS encryption. It is supported for the following
connections:
- Client to load balancer: Supported automatically on regional external Application Load Balancers;
no special configuration is required.
> [!IMPORTANT]
> **Important:** For the client to load balancer connection, H2C is not supported on global external Application Load Balancers or classic Application Load Balancers.
- Load balancer to its backends: Supported by setting the
[backend service protocol to `H2C`](https://docs.cloud.google.com/sdk/gcloud/reference/compute/backend-services/create.md.txt#--protocol).
> [!IMPORTANT]
> **Important:** For the load balancer to backend connection, H2C is not supported on classic Application Load Balancers.
H2C support is also available for load balancers created using the
GKE Gateway controller and Cloud Service Mesh.
## HTTP/3 support
[HTTP/3](https://datatracker.ietf.org/doc/rfc9114/) is a
next-generation internet protocol. It is built on top of [IETF
QUIC](https://datatracker.ietf.org/doc/rfc9000/), a protocol
developed from the original [Google
QUIC](https://www.chromium.org/quic) protocol. HTTP/3 is
supported between the external Application Load Balancer, Cloud CDN, and clients.
Google QUIC (also known as gQUIC) support was removed in April 2023 in favor
of HTTP/3 over IETF QUIC, which is both an IETF standard and has been shown to
outperform gQUIC.
Specifically:
- IETF QUIC is a transport layer protocol that provides congestion control and reliability similar to TCP, uses TLS 1.3 for security, and improved performance.
- HTTP/3 is an application layer built on top of IETF QUIC, and it relies on QUIC to handle multiplexing, congestion control, loss detection, and retransmission.
- HTTP/3 allows faster client connection initiation, eliminates head-of-line blocking in multiplexed streams, and supports connection migration when a client's IP address changes.
- HTTP/3 is supported for connections between clients and the load balancer, not connections between the load balancer and its backends.
- HTTP/3 connections use the [BBR](https://cloud.google.com/blog/products/networking/tcp-bbr-congestion-control-comes-to-gcp-your-internet-just-got-faster) congestion control algorithm.
HTTP/3 on your load balancer can improve web page load times, reduce video
rebuffering, and improve throughput on higher latency connections.
The following table specifies the HTTP/3 support for external Application Load Balancers in each
mode.
| Load balancer mode | HTTP/3 support |
|---|---|
| Global external Application Load Balancer (always Premium Tier) | Yes |
| Classic Application Load Balancer in Premium Tier | Yes |
| Classic Application Load Balancer in Standard Tier | No |
| Regional external Application Load Balancer (Premium or Standard Tier) | No |
### How HTTP/3 is negotiated
When HTTP/3 is enabled, the load balancer advertises this support to clients,
allowing clients that support HTTP/3 to attempt to establish HTTP/3 connections
with the HTTPS load balancer.
- Properly implemented clients always fall back to HTTPS or HTTP/2 when they can't establish an HTTP/3 connection.
- Clients that support HTTP/3 use their cached prior knowledge of HTTP/3 support to save unnecessary round trips in the future.
- Because of this fallback, enabling or disabling HTTP/3 in the load balancer doesn't disrupt the load balancer's ability to connect to clients.
Support is advertised in the
[`Alt-Svc`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Alt-Svc)
HTTP response header. When HTTP/3 is enabled, responses from the load balancer
include the following `alt-svc` header value:
```
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000"
```
> [!NOTE]
> **Note:** The `Alt-Svc` header advertises multiple versions of HTTP/3 in order to support earlier drafts that are used by some clients---for example, `h3-29`.
If HTTP/3 has been explicitly set to `DISABLE`, responses don't include an
`alt-svc` response header.
When you have HTTP/3 enabled on your HTTPS load balancer, some circumstances can
cause your client to fall back to HTTPS or HTTP/2 instead of negotiating HTTP/3.
These include the following:
- When a client supports versions of HTTP/3 that aren't compatible with the HTTP/3 versions supported by the HTTPS load balancer.
- When the load balancer detects that UDP traffic is blocked or rate-limited in a way that prevents HTTP/3 from working.
- The client doesn't support HTTP/3 at all, and thus doesn't attempt to negotiate an HTTP/3 connection.
When a connection falls back to HTTPS or HTTP/2, we don't count this as a
failure of the load balancer.
Before you enable HTTP/3, ensure that the previously described behaviors are
acceptable for your workloads.
### Configure HTTP/3
Both `NONE` (the default) and `ENABLE` enable HTTP/3 support for your load
balancer.
When HTTP/3 is enabled, the load balancer advertises it to clients, which allows
clients that support it to negotiate an HTTP/3 version with the load balancer.
Clients that don't support HTTP/3 don't negotiate an HTTP/3 connection. You
don't need to explicitly disable HTTP/3 unless you have identified broken client
implementations.
External Application Load Balancers provide three ways to configure HTTP/3 as shown in the
following table.
| quicOverride value | Behavior |
|---|---|
| **`NONE`** | Support for HTTP/3 is advertised to clients. |
| **`ENABLE`** | Support for HTTP/3 is advertised to clients. |
| **`DISABLE`** | Explicitly disables advertising HTTP/3 and Google QUIC to clients. |
To explicitly enable (or disable) HTTP/3, follow these steps.
### Console: HTTPS
> [!NOTE]
> **Note:** Setting the HTTP/3 negotiation isn't currently supported on the target proxies page and must be configured by editing the load balancer configuration.
1. In the Google Cloud console, go to the **Load balancing** page.
[Go to Load balancing](https://console.cloud.google.com/networking/loadbalancing/add)
2. Select the load balancer that you want to edit.
3. Click **Frontend configuration**.
4. Select the frontend IP address and port that you want to edit. To edit
an HTTP/3 configuration, the protocol must be HTTPS.
**Enable HTTP/3**
1. Select the **QUIC negotiation** menu.
2. To explicitly enable HTTP/3 for this frontend, select **Enabled**.
3. If you have multiple frontend rules representing IPv4 and IPv6, make sure to enable HTTP/3 on each rule.
**Disable HTTP/3**
1. Select the **QUIC negotiation** menu.
2. To explicitly disable HTTP/3 for this frontend, select **Disabled**.
3. If you have multiple frontend rules representing IPv4 and IPv6, make sure to disable HTTP/3 for each rule.
### gcloud: HTTPS
Before you run this command, you must create an SSL certificate resource for
each certificate.
```
gcloud compute target-https-proxies create HTTPS_PROXY_NAME \
--global \
--quic-override=QUIC_SETTING
```
Replace `QUIC_SETTING` with one of the following:
- `NONE` (default): allows Google to control when HTTP/3 is advertised.
When you select `NONE`, HTTP/3 is advertised to clients, but Google
QUIC isn't advertised. In the Google Cloud console, this option is
called **Automatic (Default)**.
- `ENABLE`: advertises HTTP/3 to clients.
- `DISABLE`: doesn't advertise HTTP/3 to clients.
### API: HTTPS
```
POST https://www.googleapis.com/v1/compute/projects/PROJECT_ID/global/targetHttpsProxies/TARGET_PROXY_NAME/setQuicOverride
{
"quicOverride": QUIC_SETTING
}
```
Replace `QUIC_SETTING` with one of the following:
- `NONE` (default): Allows Google to control when HTTP/3 is
advertised.
When you select `NONE`, HTTP/3 is advertised to clients, but Google
QUIC isn't advertised. In the Google Cloud console, this option is
called **Automatic (Default)**.
- `ENABLE`: Advertises HTTP/3 and Google QUIC to clients.
- `DISABLE`: Doesn't advertise HTTP/3 or Google QUIC to clients.
## WebSocket support
Google Cloud HTTP(S)-based load balancers support the WebSocket protocol
when you use HTTP/1.1 (HTTP or HTTPS) or HTTP/2 as the protocol to the backend.
The load balancer doesn't require any configuration to proxy WebSocket
connections.
The WebSocket protocol provides a full-duplex communication channel between
clients and the load balancer. For
more information, see [RFC 6455](https://tools.ietf.org/html/rfc6455).
For *HTTP/1.1 connections* , the WebSocket protocol uses the HTTP/1.1 `Upgrade`
mechanism to transition a TCP connection from HTTP into a WebSocket connection.
The WebSocket protocol over HTTP/1.1 works as follows:
1. The load balancer recognizes a WebSocket `Upgrade` request from an HTTP or HTTPS client. The request contains the `Connection: Upgrade` and `Upgrade: websocket` headers, followed by other relevant WebSocket-related request headers.
2. The backend sends a WebSocket `Upgrade` response. This consists of a `101 Switching Protocols` response with `Connection: Upgrade` and `Upgrade: websocket` headers, and other WebSocket-related response headers.
3. The load balancer proxies bidirectional traffic for the duration of the current connection.
If the backend instance returns a status code `HTTP 426` or `HTTP 502`,
the load balancer closes the connection.
For *HTTP/2 connections* , the WebSocket protocol uses the extended `CONNECT` method
along with pseudo-header fields (such as `:protocol`) to create a tunnel on a
single HTTP/2 stream that can carry data for WebSockets ([RFC 8441](https://datatracker.ietf.org/doc/html/rfc8441)). The WebSocket protocol over HTTP/2 works as follows:
1. The backend server sends an HTTP/2 `SETTINGS` frame containing the
`SETTINGS_ENABLE_CONNECT_PROTOCOL` parameter set to `1`. This advertises to
the load balancer that the backend supports the extended `CONNECT` method.
2. Upon receiving this setting, the load balancer recognizes it and can forward
WebSocket upgrade requests. It sends the upgrade request to the backend using
the extended `CONNECT` method. The request contains the
`:method: CONNECT` and `:protocol: websocket` pseudo-headers, along with other
relevant WebSocket-related request headers.
3. The backend sends a WebSocket success response. The backend server sends a
`200 OK` response (instead of the `101 Switching Protocols` response used in
HTTP/1.1) to establish the WebSocket tunnel over a single stream of
an HTTP/2 connection.
4. The load balancer proxies bidirectional traffic over the established HTTP/2
stream.
If the backend instance returns a status code `426` or `502`,
no WebSocket upgrade occurs, and the status code is propagated to the client.
WebSocket connection timeouts depend on the type of load balancer (global, regional, or classic). For details, see [Backend service
timeout](https://docs.cloud.google.com/load-balancing/docs/https/request-distribution#timeout-bes).
Session affinity for WebSockets works the same as for any other request.
For more information, see [Session
affinity](https://docs.cloud.google.com/load-balancing/docs/backend-service#session_affinity).
## gRPC support
[gRPC](https://grpc.io/docs/) is an open-source framework
for remote procedure calls. It is based on the HTTP/2 standard. Use cases for
gRPC include the following:
- Low-latency, highly scalable, distributed systems
- Developing mobile clients that communicate with a cloud server
- Designing new protocols that must be accurate, efficient, and language-independent
- Layered design to enable extension, authentication, and logging
To use gRPC with your Google Cloud applications, you must proxy requests
end-to-end over HTTP/2. To do this, you create an Application Load Balancer with
one of the following configurations:
- **HTTP/2 over TLS between the client and the load balancer
and H2C between the load balancer and the backend** : you create an HTTPS load
balancer (configured with a target HTTPS proxy and SSL certificate).
Additionally, you configure
the load balancer to use HTTP/2 for unencrypted connections between the load
balancer and its backends by setting the backend service protocol to `H2C`.
- **End-to-end encrypted traffic using HTTP/2 over TLS**: you create an HTTPS load
balancer (configured with a target HTTPS proxy and SSL certificate). The load
balancer negotiates HTTP/2 with clients as part of the SSL handshake by using
the ALPN TLS extension.
Additionally, you must make sure that the [backends can handle TLS
traffic](https://docs.cloud.google.com/load-balancing/docs/ssl-certificates/encryption-to-the-backends) and
configure the load balancer to use HTTP/2 for encrypted connections between
the load balancer and its backends by setting the [backend service
protocol](https://docs.cloud.google.com/sdk/gcloud/reference/compute/backend-services/create#--protocol) to
`HTTP2`.
The load balancer might still negotiate HTTPS with some clients or
accept unsecured HTTP requests on a load balancer that is configured to use
HTTP/2 between the load balancer and the backend instances. Those HTTP or
HTTPS requests are transformed by the load balancer to proxy the requests over
HTTP/2 to the backend instances.
If you want to configure an Application Load Balancer by using HTTP/2 with
Google Kubernetes Engine Ingress or by using gRPC and HTTP/2 with Ingress, see [HTTP/2
for load balancing with Ingress](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/ingress-http2).
If you want to configure an external Application Load Balancer by using HTTP/2 with
Cloud Run,
see [Use HTTP/2 behind a load balancer](https://docs.cloud.google.com/run/docs/configuring/http2.md.txt#http2-lb).
For information about troubleshooting problems with HTTP/2, see [Troubleshooting
issues with HTTP/2 to the backends](https://docs.cloud.google.com/load-balancing/docs/https/troubleshooting-ext-https-lbs).
For information about HTTP/2 limitations, see [HTTP/2
limitations](https://docs.cloud.google.com/load-balancing/docs/https#HTTP2-limitations).
If you are using bidirectional gRPC streaming and the client connects to a
global external Application Load Balancer, the backend server might hang if a client performs an
immediate half-close (sends an `END_STREAM` flag immediately after stream
creation without sending any data). This occurs due to a conflict between HTTP/2
optimization and gRPC framing specifications. To learn more,
see [Troubleshoot gRPC issues](https://docs.cloud.google.com/load-balancing/docs/https/troubleshooting-ext-https-lbs.md.txt#troubleshoot-grpc-issues).
## TLS support
By default, an HTTPS target proxy accepts only TLS 1.0, 1.1, 1.2, and 1.3 when
terminating client SSL requests.
When the global external Application Load Balancer or the regional external Application Load Balancer use HTTPS as the backend service protocol, they can negotiate TLS 1.2 or 1.3 to the backend.
When the classic Application Load Balancer uses HTTPS as the backend service protocol, it
can negotiate TLS 1.0, 1.1, 1.2, or 1.3 to the backend.
### Mutual TLS support
Mutual TLS, or mTLS, is an industry standard protocol for mutual authentication
between a client and a server. mTLS helps ensure that both the client and server
authenticate each other by verifying that each holds a valid certificate issued
by a trusted certificate authority (CA). Unlike standard TLS, where only the
server is authenticated, mTLS requires both the client and server to present
certificates, confirming the identities of both parties before communication is
established.
All of the Application Load Balancers support mTLS. With mTLS, the load balancer
requests that the client send a certificate to authenticate itself during the
TLS handshake with the load balancer. You can configure a
[Certificate Manager](https://docs.cloud.google.com/certificate-manager/docs/overview.md.txt) trust
store that the load balancer then uses to validate the client certificate's
chain of trust.
For more information about mTLS, see [Mutual TLS
authentication](https://docs.cloud.google.com/load-balancing/docs/mtls).
## TLS 1.3 early data support
> [!NOTE]
> **Note:** TLS 1.3 early data or 0-RTT is only supported for resuming previous sessions between the clients and the Cloud CDN or load balancer. Cloud CDN or load balancer cannot use TLS early data when forwarding requests to the origin servers.
TLS 1.3 early data is supported on the target HTTPS proxy of the following
external Application Load Balancers for both HTTPS over TCP (HTTP/1.1, HTTP/2) and HTTP/3
over QUIC:
- Global external Application Load Balancers
- Classic Application Load Balancers
TLS 1.3 was defined in [RFC
8446](https://datatracker.ietf.org/doc/html/rfc8446) and introduces the concept
of [early data](https://datatracker.ietf.org/doc/html/rfc8470), also known as
*zero-round-trip time (0-RTT) data*, which can improve application performance
for resumed connections by 30 to 50%.
With TLS 1.2, two round trips are required before data can be securely
transmitted. TLS 1.3 reduces this to one round trip (1-RTT) for new connections,
allowing clients to send application data immediately after the first server
response. Additionally, TLS 1.3 introduces the concept of early data for resumed
sessions, enabling clients to send application data with the initial
`ClientHello`, thereby reducing the effective round-tip time to *zero* (0-RTT).
TLS 1.3 early data allows the backend server to begin processing client data
before the handshake process with the client is complete, thereby reducing
latency; however, care must be taken to mitigate replay risks.
Because early data is sent before the handshake is complete, an attacker can
attempt to capture and replay requests. To mitigate this, the backend server
must carefully control early data usage, limiting its use to idempotent
requests. HTTP methods that are intended to be idempotent but which might
trigger nonidempotent changes---such as a GET request modifying a
database---must not accept early data. In such cases, the backend server
must reject requests with the HTTP `Early-Data: 1` header by returning an HTTP
`425 Too Early` status code.
Requests with early data have the [HTTP Early-Data header](https://datatracker.ietf.org/doc/html/rfc8470#section-5.1) set to a value of
`1`, which indicates to the backend server that the request has been conveyed in
TLS early data. It also indicates that the client understands the [HTTP `425 Too
Early` status code](https://datatracker.ietf.org/doc/html/rfc8470#section-5.2).
### TLS early data (0-RTT) modes
You can configure TLS early data using one of the following modes on the target
HTTPS proxy resource of the load balancer.
- `STRICT`. This enables TLS 1.3 early data for requests with safe HTTP
methods (GET, HEAD, OPTIONS, TRACE), and HTTP requests that don't have query
parameters. Requests that send early data containing nonidempotent HTTP
methods (such as POST or PUT) or with query parameters are rejected with an
HTTP `425` status code.
- `PERMISSIVE`. This enables TLS 1.3 early data for requests with safe HTTP
methods (GET, HEAD, OPTIONS, TRACE). This mode doesn't deny requests
that include query parameters. The application owner must ensure that
early data is safe to use for each request path, particularly for
endpoints where request replay might cause unintended side effects, such
as logging or database updates triggered by GET requests.
- `DISABLED`. TLS 1.3 early data isn't advertised, and any (invalid)
attempts to send early data are rejected. If your applications aren't
equipped to handle early data requests safely, you must disable early
data. TLS 1.3 early data is disabled by default.
- `UNRESTRICTED` (not recommended for most workloads). This enables TLS 1.3
early data for requests with any HTTP method including nonidempotent
methods, such as POST. This mode doesn't enforce any other limitations.
This mode can be valuable for gRPC use cases. However, we don't recommend
this method unless you have evaluated your security posture and mitigated
the risk of replay attacks using other mechanisms.
### Configure TLS early data
To explicitly enable or disable TLS early data, do the following:
### Console
1. In the Google Cloud console, go to the **Load balancing** page.
[Go to Load balancing](https://console.cloud.google.com/net-services/loadbalancing/list)
2. Select the load balancer for which you need to enable early data.
3. Click **Edit**.
4. Click **Frontend configuration**.
5. Select the frontend IP address and port that you want to edit. To enable
TLS early data, the protocol must be HTTPS.
6. In the **Early data (0-RTT)** list, select a TLS early data mode.
7. Click **Done**.
8. To update the load balancer, click **Update**.
### gcloud
1. Configure the TLS early data mode on the target HTTPS proxy of an
Application Load Balancer.
```
gcloud compute target-https-proxies update TARGET_HTTPS_PROXY \
--tls-early-data=TLS_EARLY_DATA_MODE
```
Replace the following:
- `TARGET_HTTPS_PROXY`: the target HTTPS proxy of your load balancer
- `TLS_EARLY_DATA_MODE`: `STRICT`, `PERMISSIVE`, `DISABLED`, or `UNRESTRICTED`
### API
```
PATCH https://compute.googleapis.com/compute/v1/projects/{project}/global/targetHttpsProxies/TARGET_HTTPS_PROXY
{
"tlsEarlyData":"TLS_EARLY_DATA_MODE",
"fingerprint": "FINGERPRINT"
}
```
Replace the following:
- `TARGET_HTTPS_PROXY`: the target HTTPS proxy of your load balancer
- `TLS_EARLY_DATA_MODE`: `STRICT`, `PERMISSIVE`, `DISABLED`, or `UNRESTRICTED`
- `FINGERPRINT`: a Base64 encoded string. An up-to-date fingerprint must be provided in order to patch the target HTTPS proxy; otherwise, the request fails with an HTTP `412 Precondition Failed` status code.
After you have configured TLS early data, you can issue requests from an HTTP
client that supports TLS early data. You can observe lower latency for
resumed requests.
If a non-RFC-compliant client sends a request with a nonidempotent method or
with query parameters, the request is denied. You see an HTTP `425 Early` status
code in the load balancer logs and the following HTTP response:
```
HTTP/1.1 425 Too Early
Content-Type: text/html; charset=UTF-8
Referrer-Policy: no-referrer
Content-Length: 1558
Date: Thu, 03 Aug 2024 02:45:14 GMT
Connection: close
Error 425 (Too Early)
The request was sent to the server too early, please retry. That's
all we know.
```
## Google tag gateway for advertisers
[Google tag gateway](https://developers.google.com/tag-platform/tag-manager/gateway)
lets website owners host and deploy Google tags through
Google Cloud. These Google tags help you analyze traffic on
your website. You can use a global external Application Load Balancer to route measurement traffic
on your website through your domain for improved measurement data accuracy. This
provides more reliable data for advertising campaign optimization.
Note that Google tag gateway is only supported on global external Application Load Balancers. If you
use a classic Application Load Balancer,
[migrate your resources from classic to global external Application Load Balancer](https://docs.cloud.google.com/load-balancing/docs/https/migrate-from-classic-global.md.txt).
Configuration and management of the Google tag gateway are managed outside of
the Google Cloud console. You must configure this feature using
the Google Tag Manager interface.
- In the Google Cloud console, the **Load balancer details** page displays a read-only visual indicator showing that Google tag gateway is enabled.
- You can't edit or delete the Google tag gateway configuration from the Google Cloud console.
For complete instructions on how to set up the Google tag gateway for
advertisers, see
[Set up Google tag gateway for advertisers](https://developers.google.com/tag-platform/tag-manager/gateway/setup-guide?setup=auto&provider=google-cloud#provider).
While configuration is managed with Google Ads, request processing and Google
tag injections performed by the gateway are recorded in your
global external Application Load Balancer's logging and monitoring. For more information, see
[Global external Application Load Balancer logging and monitoring](https://docs.cloud.google.com/load-balancing/docs/https/https-logging-monitoring.md.txt).
For information about troubleshooting issues with Google tag gateways, see
[Troubleshoot Google tag gateway injection issues](https://docs.cloud.google.com/load-balancing/docs/https/troubleshooting-ext-https-lbs.md.txt#troubleshoot-gtg-injection).
If you encounter issues during setup that the documentation can't resolve, contact
[Google Ads Support](https://support.google.com/tagmanager/answer/14681508).
## Limitations
- HTTPS load balancers don't send a [`close_notify` closure
alert](https://tools.ietf.org/html/rfc5246#section-7.2.1) when terminating
SSL connections. That is, the load balancer closes the TCP connection
instead of performing an SSL shutdown.
- HTTPS load balancers support only lowercase characters in domains in a
common name (`CN`) attribute or a subject alternative name (`SAN`) attribute
of the certificate. Certificates with uppercase characters in domains are
returned only when set as the [primary
certificate](https://docs.cloud.google.com/load-balancing/docs/ssl-certificates#multiplessl) in the
target proxy.
- HTTPS load balancers don't use the Server Name Indication (SNI) extension
when connecting to the backend, except for load balancers with [Internet NEG
backends](https://docs.cloud.google.com/load-balancing/docs/negs/internet-neg-concepts#support_for_the_ssl_server_name_indication_sni_extension).
For more information, see [Encryption from the load balancer to the
backends](https://docs.cloud.google.com/load-balancing/docs/ssl-certificates/encryption-to-the-backends).
- Google Cloud doesn't guarantee that an underlying TCP connection can
remain open for the entirety of the value of the backend service timeout.
Client systems must implement retry logic instead of relying on a TCP
connection to be open for long periods of time.
- When you create a regional external Application Load Balancer in Premium Tier using the
Google Cloud console, only [regions
supporting Standard Tier](https://docs.cloud.google.com/network-tiers/docs/overview#regions_supporting_standard_tier) are available in the Google Cloud console.
For access to other regions, use either the gcloud CLI or the
API.
- The GFE proxies used by global and classic Application Load Balancers don't support
*early* `200 OK` responses that are sent before the request's POST payload
has been fully proxied to the backend. Sending an early `200 OK` response
causes the GFE to close the connection to the backend.
Your backend must respond with `100 Continue` responses after the request
headers are received, and then wait until the request's POST payload has been
fully proxied before responding with the final `200 OK` response code.
- When using regional external Application Load Balancers with Cloud Run in
a Shared VPC environment, standalone VPC networks in
service projects can send traffic to any other
Cloud Run services deployed in any other service
projects within the same Shared VPC environment. This is a known
issue.
- Cloud CDN lets you force an object or set of objects to be
ignored by the cache by requesting a [cache
invalidation](https://docs.cloud.google.com/cdn/docs/cache-invalidation-overview). When you're using a
global external Application Load Balancer with [Shared VPC cross-project service
referencing](https://docs.cloud.google.com/load-balancing/docs/https#shared-vpc), by default, service project admins won't have the
required permissions to request cache invalidations. This is because cache
invalidation is configured in the frontend project (that is, the project
that has the forwarding rule, target proxy, and URL map of the load
balancer). Thus, cache invalidations can only be issued by principals who
have the IAM roles for configuring load balancer related resources in the
frontend projects (for example, the Compute Network Admin role). Service
admins, who control provisioning of the backend services in a separate
project, need to work with the load balancer administrator of the frontend
project to issue cache invalidation for their cross-project services.
- When using a classic Application Load Balancer, the load balancer automatically adds an
HTTP `Content-Length: 0` header to `DELETE` requests over HTTP/1.1 and
HTTP/2, even if the original request did not specify a content length. This
behavior might cause Cloud Armor security policies to block these
requests if your policy is configured to filter or reject requests
containing a content-length header. This issue is specific to a
classic Application Load Balancer and doesn't occur with `GET` or `HEAD` requests.
## What's next
- To learn how external Application Load Balancers handle connections, route traffic,
and maintain session affinity, see [Request distribution for
external Application Load Balancers](https://docs.cloud.google.com/load-balancing/docs/https/request-distribution).
- To learn how to deploy a global external Application Load Balancer, see [Setting up an
external Application Load Balancer with a Compute Engine
backend](https://docs.cloud.google.com/load-balancing/docs/https/ext-http-lb-simple).
- To learn how to deploy a regional external Application Load Balancer, see [Setting up a
regional external Application Load Balancer with a Compute Engine
backend](https://docs.cloud.google.com/load-balancing/docs/https/setting-up-reg-ext-https-lb).
-
If you are an existing user of the classic Application Load Balancer, make sure that you
review [Migration
overview](https://docs.cloud.google.com/load-balancing/docs/https/migrate-to-global) when you plan a new deployment with the global external Application Load Balancer.
- To learn how to automate your external Application Load Balancer setup with Terraform, see
[Terraform module examples for
external Application Load Balancers](https://docs.cloud.google.com/load-balancing/docs/https/ext-http-lb-tf-module-examples).
- To learn how to configure advanced traffic management capabilities available
with the global external Application Load Balancer, see [Traffic management overview for
global external Application Load Balancers](https://docs.cloud.google.com/load-balancing/docs/https/traffic-management-global).
- To learn how to configure failover for global external Application Load Balancers, see
[Failover for
global external Application Load Balancers](https://docs.cloud.google.com/load-balancing/docs/https/global-applb-failover-overview).
- To learn how to configure advanced traffic management capabilities
available with the regional external Application Load Balancer, see [Traffic management
overview for regional external Application Load Balancers](https://docs.cloud.google.com/load-balancing/docs/https/traffic-management-regional).
- To learn how to configure high availability for regional external Application Load Balancers, see
[High availability for
regional external Application Load Balancers](https://docs.cloud.google.com/load-balancing/docs/https/reg-applb-ha-overview).
- To review best practices for performance, high availability, and failover, see
[External Application Load Balancer best
practices](https://docs.cloud.google.com/load-balancing/docs/https/http-load-balancing-best-practices).
- To learn about serving websites, see [Serving
websites](https://docs.cloud.google.com/solutions/web-serving-overview).
- To find the locations for Google PoPs, see
[GFE locations](https://docs.cloud.google.com/load-balancing/docs/locations.md.txt).
- To learn about capacity management, see [Capacity management with load
balancing
tutorial](https://docs.cloud.google.com/load-balancing/docs/tutorials/capacity-management-with-load-balancing)
and [Application capacity optimizations with global load
balancing](https://docs.cloud.google.com/load-balancing/docs/tutorials/about-capacity-optimization-with-global-lb).
- To learn how to use Certificate Manager to provision and
manage SSL certificates, see the [Certificate Manager
overview](https://docs.cloud.google.com/certificate-manager/docs/overview).
- To insert custom logic into the load balancing data path, configure
[Service Extensions plugins or callouts](https://docs.cloud.google.com/service-extensions/docs/overview.md.txt#integration-lb).
- For regional external Application Load Balancers only, you can try using [Apigee Shadow API
Discovery](https://docs.cloud.google.com/apigee/docs/api-observation/shadow-api-discovery) to find
shadow APIs (also known as undocumented APIs) in your existing
Google Cloud infrastructure. Make sure that you read the associated
[limitations](https://docs.cloud.google.com/apigee/docs/api-observation/shadow-api-discovery.md.txt#behaviors-and-limitations)
before you enable Shadow API Discovery.