This page explains concepts about how external passthrough Network Load Balancers distribute traffic.
Backend selection and connection tracking
Backend selection and connection tracking work together to balance multiple connections across different backends and to route all packets for each connection to the same backend. This is accomplished with a two-part strategy. First, a backend is selected using consistent hashing. Then, this selection is recorded in a connection tracking table.
The following steps describe backend selection and connection tracking.
1. Check for a connection tracking table entry
The load balancer determines whether a load-balanced packet belongs to a new connection or to an existing connection by using the following process:
TCP packet with the
SYNflag:If the load balancer's connection tracking mode is
PER_CONNECTION, continue to the Identify eligible backends step. InPER_CONNECTIONtracking mode, a TCP packet with theSYNflag always represents a new connection, regardless of the configured session affinity.If the load balancer's connection tracking mode is
PER_SESSIONand the session affinity is eitherNONEorCLIENT_IP_PORT_PROTO, continue to the Identify eligible backends step. InPER_SESSIONtracking mode, a TCP packet with theSYNflag represents a new connection only when using one of the 5-tuple session affinity options (NONEorCLIENT_IP_PORT_PROTO).
- Connection tracking not supported: if the configured session affinity doesn't support connection tracking for the packet's protocol, continue to the Identify eligible backends step. For information about which protocols are connection trackable, see the table in the Connection tracking mode section.
All other packets: the load balancer checks if the packet matches an existing connection tracking table entry. The granularity of the packet hash used to check for an existing connection tracking table entry depends on the connection tracking mode and session affinity you configured. For more information, see the table in the Connection tracking mode section.
If the packet matches a connection tracking table entry, the load balancer sends the packet to the previously selected backend.
If the packet doesn't match a connection tracking table entry, continue to the Identify eligible backends step.
For information about how long a connection tracking table entry persists and under what conditions it persists, see the Manage connection tracking table entries step.
2. Backend selection steps
For a new connection, the load balancer uses consistent hashing to select a backend from among the eligible backends.
The following steps outline the process to select an eligible backend for a new connection and then record that connection in a connection tracking table.
2.1 Identify eligible backends
Eligible backends are the backends that are candidates to receive new connections. The following table defines the set of eligible backends depending on whether you've configured a failover policy and whether weighted load balancing is enabled.
| Failover policy | Weighted load balancing | Eligible backends |
|---|---|---|
When no failover policy is configured and weighted load balancing is disabled, all configured backends are primary backends. The set of eligible backends is defined as follows:
|
||
When no failover policy is configured and weighted load balancing is enabled, the eligible backends are those that come from the first of the following sets that isn't empty:
|
||
When a failover policy is configured and weighted load balancing is disabled, the load balancer uses health check information and failover policy configuration to define the set of eligible backends:
|
||
When a failover policy is configured and weighted load balancing is enabled, the load balancer uses weight information, health check information, and failover policy configuration to define the set of eligible backends:
|
2.2 Select an eligible backend
The load balancer maintains hashes of eligible backends, with each backend hash mapped to a unit circle. Weighted load balancing alters how eligible backends are mapped to the circle such that backends with higher weights are more likely to be selected, proportional to their weights.
When processing a packet for a connection that's not in the connection tracking table, the load balancer computes a hash of the packet characteristics and maps that hash to the same unit circle, selecting an eligible backend on the circle's circumference. The set of packet characteristics used to calculate the packet hash is defined by the session affinity setting. For example, when the selected session affinity results in a 2-tuple or 3-tuple backend selection hash, all TCP connections from a source IP address are mapped to the same eligible backend.
- If a session affinity isn't explicitly configured, the
NONEsession affinity is the default.
Consistent hashing ensures the load balancer assigns new connections to eligible backends in a way that minimizes mapping disruptions even if the number of eligible backends or their weights change.
The load balancer always selects the same eligible backend for a connection, and more generally, always selects the same eligible backend for all packets with identical packet characteristics as defined by the session affinity setting, in the following situations:
If weighted load balancing isn't configured, when the set of eligible backends does not change.
If weighted load balancing is configured, when the set of eligible backends does not change, and the weight of each eligible backend remains constant.
If an eligible backend is added, removed, or has its weight changed, consistent hashing aims to minimize the disruption of mappings to the other eligible backends—that is, most connections that map to other eligible backends continue to map to the same eligible backend.
Furthermore, consistent hashing ensures the load balancer distributes new connections among eligible backends as fairly as possible. For all possible packet hashes as defined by the configured session affinity setting (and more specifically, for all possible connections when the session affinity results in a 5-tuple hash for backend selection):
If weighted load balancing isn't configured, approximately
1/Npossible packet hashes map to each eligible backend, whereNis the count of eligible backends.If weighted load balancing is configured, the ratio of possible packet hashes that map to each eligible backend is approximately: the weight of an eligible backend divided by the sum of all eligible backend weights.
The following two examples show how weighted load balancing affects the selection probabilities of each eligible backend:
If the backend service has two eligible backends—the first having weight
1and the second having weight4—the first eligible backend has a 20% (1÷(1+4)) selection probability, and the second eligible backend has an 80% (4÷(1+4)) selection probability.If the backend service has three eligible backends—eligible backend
ahaving weight0, eligible backendbhaving weight2, and eligible backendchaving weight6—backendahas a 0% (0÷(0+2+6)) selection probability, backendbhas a 25% (2÷(0+2+6)) selection probability, and backendchas a 75% (6÷(0+2+6)) selection probability.
2.3 Create a connection tracking table entry
After selecting a backend, the load balancer creates a connection tracking table entry if the configured session affinity supports connection tracking for the packet's protocol.If the configured session affinity doesn't support connection tracking for the packet's protocol, skip this step.
If the configured session affinity supports connection tracking for the packet's protocol, the connection tracking table entry that's created maps packet characteristics to the selected backend. The packet header fields used for this mapping depend on the connection tracking mode and session affinity you configured.
For information about which protocols are connection trackable based on your configuration choices, and what packet characteristics are used for the hash in the connection tracking table, see the table in the Connection tracking mode section.
3. Manage connection tracking table entries
The load balancer manages connection tracking table entries according to the following events and rules:
- Idle entries are removed: a connection tracking table entry is removed after the connection has been idle for 60 seconds. For more information, see Idle timeout.
Closed TCP connections: connection tracking table entries for TCP connections are not removed when a TCP connection is closed with a
FINorRSTpacket. They might be removed later as an idle entry. Each new TCP connection always carries theSYNflag, and is subject to the processing described in the Check for a connection tracking table entry step.Connection draining on failover: when at least one failover backend is configured and the connection draining on failover setting is disabled, the load balancer removes all entries in the connection tracking table when the set of eligible backends switches between primary and failover backends. For more information, see Connection draining on failover.
Connection persistence on unhealthy backends: entries in the connection tracking table can be removed if a backend becomes unhealthy. This behavior depends on factors described in Connection persistence on unhealthy backends.
When a connection tracking table entry is removed because a previously selected backend changes from healthy to unhealthy, subsequent packets for the connection are treated as if they belong to a new connection. After selecting a new eligible backend for the subsequent packets, the load balancer creates a replacement connection tracking table entry.
A replacement connection tracking table entry behaves exactly like any other connection tracking table entry, and is subject to the events and rules of this step.
If the previously selected backend returns to healthy from unhealthy, the health check change alone doesn't cause the replacement connection tracking table entry to be removed. An exception happens when at least one failover backend is configured and the connection draining on failover setting is disabled; if the change in health check state of a previously selected backend coincides with the set of eligible backends switching between primary and failover backends, connection tracking table entries are removed.
Connection draining for removed, stopped, or deleted backends: if connection draining for removed, stopped, or deleted backends is enabled, connection tracking table entries are removed after a configurable connection draining timeout. Counting to the timeout begins when the command to remove, stop, or delete a backend is received. If connection draining for removed, stopped, or deleted backends is disabled, connection tracking table entries are removed when the command to remove, stop, or delete a backend is received. For more information, see Enable connection draining.
Session affinity
The session affinity setting of an external passthrough Network Load Balancer defines the packet hash for backend selection, and, based on the connection tracking mode, the packet hash for connection tracking.
You configure session affinity on the backend service, not on each backend instance group or NEG. The session affinity determines which IP and Layer 4 headers are used to calculate a hash of packet characteristics. The hash of packet characteristics is used in the Backend selection steps.
External passthrough Network Load Balancers support the following session affinity settings.
| Hash method for backend selection | Session affinity setting |
|---|---|
|
5-tuple hash (consists of source IP address, source port, protocol, destination IP address, and destination port) for non-fragmented packets that include port information (TCP packets and non-fragmented UDP packets) OR3-tuple hash (consists of source IP address, destination IP address, and protocol) for fragmented UDP packets and packets of all other protocols |
NONE1
OR CLIENT_IP_PORT_PROTO
|
| 3-tuple hash (consists of source IP address, destination IP address, and protocol) |
CLIENT_IP_PROTO |
| 2-tuple hash (consists of source IP address and destination IP address) |
CLIENT_IP |
1 NONE session affinity doesn't indicate that there
is no session affinity. Instead, it means that session affinity is done
with a 5-tuple hash or a 3-tuple hash of packet characteristics—functionally
the same behavior as when CLIENT_IP_PORT_PROTO is set.
Connection tracking policy
This section describes the settings in the load balancer's connection tracking policy:
Connection tracking mode
This section describes when and how the load balancer creates entries in its connection tracking table. External passthrough Network Load Balancers support connection tracking based on protocol and session affinity:TCP connections are always connection trackable, for all session affinity options.
UDP, ESP, and GRE connections are connection trackable for all session affinity options except for
NONE.All other protocols, such as ICMP and ICMPv6, aren't connection trackable.
When connection tracking is possible, the connection tracking mode, protocol, and session affinity determine the set of packet characteristics that are used to make the packet hash in each connection tracking table entry.
The connection tracking mode can be one of the following:
PER_CONNECTION. This is the default and most granular connection tracking mode. Each connection is defined as either a 5-tuple hash or a 3-tuple hash of packet characteristics, depending on whether port information is present in the packet. Non-fragmented packets that include port information (such as TCP packets and non-fragmented UDP packets) are tracked with 5-tuple hashes. All other packets are tracked with 3-tuple hashes.PER_SESSION. This less granular connection tracking mode uses a hash that matches the session affinity hash. Depending on the chosen session affinity, thePER_SESSIONtracking mode can treat multiple distinct connections as a single connection for connection tracking purposes. This reduces the frequency that a connection is considered new and subject to the Backend selection steps.
The following table summarizes:
- The packet hashes used for backend selection; and
- The packet hashes used for connection tracking, based on the connection tracking mode, protocol, and session affinity.
| Session affinity | Packet hash for backend selection | Packet hash for connection tracking | |
|---|---|---|---|
When using PER_CONNECTION tracking mode (default) |
When using PER_SESSION tracking mode |
||
NONE (Default) |
|
|
|
CLIENT_IP_PORT_PROTO |
|
|
|
CLIENT_IP_PROTO |
|
|
|
CLIENT_IP |
|
|
|
To learn how to change the connection tracking mode, see Configure a connection tracking policy.
Connection persistence on unhealthy backends
Connection persistence on unhealthy backends controls whether existing connections persist on a previously-selected backend VM or endpoint after the backend becomes unhealthy, provided that the backend stays in a load-balanced instance group or NEG.
The following connection persistence options are available:
DEFAULT_FOR_PROTOCOL(default)NEVER_PERSISTALWAYS_PERSIST
The following table summarizes whether connections persist based on unhealthy backends, depending on the connection persistence option, session affinity, connection tracking mode, and protocol.
| Connection persistence on unhealthy backends option | Connection persistence on unhealthy backends behavior | |
|---|---|---|
When using PER_CONNECTION tracking mode (default) |
When using PER_SESSION tracking mode |
|
DEFAULT_FOR_PROTOCOL |
|
|
NEVER_PERSIST |
All protocols: connections never persist on unhealthy backends | |
ALWAYS_PERSIST
This option should only be used for advanced use cases. |
|
Configuration not possible |
When connection persistence on unhealthy backends applies to traffic, each connection persists as long as a corresponding connection tracking table entry exists. For more information, see the Manage connection tracking table entries step.
To learn how to change connection persistence behavior, see Configure a connection tracking policy.
TCP connection persistence behavior on unhealthy backends
The load balancer uses 5-tuple hash connection tracking for TCP connections in these situations:
- When using the
PER_CONNECTIONtracking mode (all session affinities), or - When using the
PER_SESSIONtracking mode, and the session affinity is eitherNONEorCLIENT_IP_PORT_PROTO.
When the load balancer uses a 5-tuple hash connection tracking for TCP connections, keep the following behaviors in mind:
- If the unhealthy backend continues to respond to packets, the connection continues until it is reset or closed (by either the unhealthy backend or the client).
- If the unhealthy backend sends a TCP reset (RST) packet or does not respond to
packets, then the client might retry with a new connection, letting the load
balancer select a different eligible backend. (TCP
SYNpackets are treated as new connections in the Identify eligible backends step.)
Idle timeout
Entries in connection tracking tables expire 60 seconds after the load balancer processes the last packet that matched the entry. This idle timeout value can't be modified.Connection draining for removed, stopped, or deleted backends
Connection draining provides a configurable minimum amount of time for existing connections to persist in the load balancer's connection tracking table when one of the following happens:
- A virtual machine (VM) instance is removed from a backend instance group (this includes abandoning an instance in a backend managed instance group)
- A VM is stopped or deleted (this includes automatic actions like rolling updates or scaling down a backend managed instance group)
- An endpoint is removed from a backend network endpoint group (NEG)
By default, connection draining when backends are removed, stopped, or deleted is disabled. For more information, see Enabling connection draining.
Weighted load balancing
Weighted load balancing influences which backends are eligible backends in the Backend selection steps. Each backend VM or endpoint reports its weight to the load balancer by using an HTTP health check and a custom response header. To use weighted load balancing, you must configure the following on the load balancer's backend service:
- The locality policy (
localityLbPolicy) must be set toWEIGHTED_MAGLEV. The health check must be an HTTP health check that sends a special response header:
- The response header's field name must be
X-Load-Balancing-Endpoint-Weight. - The response header's field values can range from
0to1000, inclusive.
- The response header's field name must be
For more information, see Configure weighted load balancing.
Considerations for weighted load balancing
Weighted load balancing is useful in the following scenarios, which allow a backend to continue processing its existing connections:
Weighted load balancing allows a backend that's processing long-running connections or connections that involve large amounts of data to tell the load balancer to reduce the number of new connections it receives.
Weighted load balancing allows a backend that's overloaded or entering maintenance to remove itself from the eligible backends for new connections. To do this, the overloaded backend sends the
X-Load-Balancing-Endpoint-Weight: 0response header (and can either continue to pass or fail the load balancer health check). This works because all nonzero weight backends (regardless of health check state) are more preferable eligible backends in the Identify eligible backends step.
Keep the following in mind when using load balancing:
If eligible backends change their weights frequently, weighted load balancing can be detrimental to session affinity. For more information, see the Select an eligible backend step.
If you use the same instance group or NEG as a backend of two or more load balancer backend services, you can report a unique weight for each backend service by using the following strategy:
Use a unique HTTP health check for each backend service. Each health check can use a unique destination port or
request-pathparameter.Configure backend instances or endpoints to respond with appropriate weight information for each health check.
Failover
Failover lets you influence the set of eligible backends for new connections by classifying each backend instance group or NEG as primary or failover.
By default, when you add an instance group or NEG to a backend service, the member VMs or endpoints are primary backends, and the instance group or NEG is a primary backend group. With failover, you can add a failover backend group (instance group or NEG) whose member VMs or endpoints are failover backends:
- Failover requires a backend service to have at least one primary backend group and at least one failover backend group.
- You can add up to 50 primary backend groups and 50 failover backend groups to a backend service.
With failover, the following factors determine the set of eligible backends:
- The health state of each backend
- The failover ratio that you've configured
- The drop traffic if backends are unhealthy setting
- Whether you are using failover by itself or in conjunction with weighted load balancing
Failover policy
When a backend service has at least one primary backend group and at least one failover backend group, you can adjust the following settings in its failover policy:
- Failover ratio: a number between
0.0and1.0, inclusive. - Drop traffic if backends are unhealthy: a boolean that determines the load balancer's last resort behavior. The failover ratio and the drop traffic if backends are unhealthy setting work together with other factors to control the set of eligible backends.
- Connection draining on failover: a boolean that controls whether connections persist on previously-selected backends when the set of eligible backends switches between primary and failover backends.
Failover ratio
The configured failover ratio determines when the set of eligible backends
switches between primary and failover backends. The
ratio can be a number between 0.0 to 1.0, inclusive. If you don't specify a
failover ratio, Google Cloud uses a default value of 0.0. It's a best
practice to set your failover ratio to a number that works for your use case
rather than relying on this default.
Connection draining on failover
Connection draining on failover controls whether an existing connection persists on a previously-selected backend VM or endpoint when the set of eligible backends switches between primary and failover backends.
Connection draining on failover is enabled by default. The following table summarizes whether connections persist, depending on the connection draining on failover option and protocol:
| Connection draining on failover option | Behavior when the set of eligible backends switches between primary and failover backends |
|---|---|
| Enabled (default) |
For information about which protocols are connection trackable, see the table in the Connection tracking mode section. |
| Disabled | All protocols: connections don't persist when the set of eligible backends switches between primary and failover backends |
Disabling connection draining on failover and failback is useful for the following scenarios:
Patching backend VMs. Prior to patching, you can configure nonzero weight healthy primary backends to fail health checks, or you can set their weights to zero. This way, eligible backends can be nonzero weight healthy failover backends. By disabling connection draining on failover and failback, the load balancer removes connection tracking table entries, applying the Backend selection steps to subsequent packets and delivering them to a different eligible backend. The different backend then closes the connection with a TCP reset, so that client VMs can quickly establish a new connection to the load balancer.
Single backend VM for data consistency. If you need to ensure that the set of eligible backends has no more than one member VM or endpoint, disabling connection draining on failover and failback reduces the possibility of data inconsistencies.
To learn how to disable connection draining on failover and failback, see Disabling connection draining on failover and failback.
Best practices and guidance
You can optimize the external passthrough Network Load Balancer by following these operational guidelines. The following sections provide technical requirements for managing fragmented UDP packets and best practices for testing load distribution from a single client.
Handling UDP fragmentation
Backend service-based external passthrough Network Load Balancers can process both fragmented and unfragmented UDP packets. If your application uses fragmented UDP packets, keep the following in mind:
- UDP packets might become fragmented before reaching a Google Cloud VPC network.
- Google Cloud VPC networks forward UDP fragments as they arrive (without waiting for all fragments to arrive).
- Non-Google Cloud networks and on-premises network equipment might forward UDP fragments as they arrive, delay fragmented UDP packets until all fragments have arrived, or discard fragmented UDP packets. For details, see the documentation for the network provider or network equipment.
If you expect fragmented UDP packets and need to route them to the same backends, use the following forwarding rule and backend service configuration parameters:
Forwarding rule configuration: Use only one
UDPorL3_DEFAULTforwarding rule per load-balanced IP address, and configure the forwarding rule to accept traffic on all ports. This ensures that all fragments arrive at the same forwarding rule. Even though the fragmented packets (other than the first fragment) lack a destination port, configuring the forwarding rule to process traffic for all ports also configures it to receive UDP fragments that have no port information. To configure all ports, either use the Google Cloud CLI to set--ports=ALLor use the API to setallPortstoTrue.Backend service configuration: Set the backend service's session affinity to
CLIENT_IP(2-tuple hash) orCLIENT_IP_PROTO(3-tuple hash) so that the same backend is selected for UDP packets that include port information and UDP fragments (other than the first fragment) that lack port information. Set the backend service's connection tracking mode toPER_SESSIONso that the connection tracking table entries are built by using the same 2-tuple or 3-tuple hashes.
Testing from a single client
When testing an external passthrough Network Load Balancer from a single client, keep the following in mind:
If the client VM is not a backend of the load balancer: new connections are processed as described in the Backend selection and connection tracking steps. In the Select an eligible backend step, the load balancer creates a hash of packet characteristics according to the session affinity.
Remember that all session affinity options rely on at least the IP address of the client, connections from the same client might be distributed to the same eligible backend more frequently than you might expect. Consequently, you can't accurately model the overall distribution of new connections by connecting to an external passthrough Network Load Balancer from a single client.
If the client VM is also a backend VM of the load balancer: new connections aren't actually processed by the load balancer at all. Outbound packets with the destination IP address of the load balancer's forwarding rule are routed locally within the guest OS of the client due to the presence of a local route for the forwarding rule.
What's next
- To configure an external passthrough Network Load Balancer with a backend service for TCP or UDP traffic only (supporting IPv4 and IPv6 traffic), see Set up an external passthrough Network Load Balancer with a backend service.
- To configure an external passthrough Network Load Balancer for multiple IP protocols (supporting IPv4 and IPv6 traffic), see Set up an external passthrough Network Load Balancer for multiple IP protocols.
- To configure an external passthrough Network Load Balancer with a zonal NEG backend, see Set up an external passthrough Network Load Balancer with zonal NEGs
- To learn how to transition an external passthrough Network Load Balancer from a target pool backend to a regional backend service, see Transitioning an external passthrough Network Load Balancer from a target pool to a backend service.