This document provides instructions for configuring a cross-region internal Application Load Balancer
for your services that run on Compute Engine virtual machine (VM) instances.

## Before you begin

Before following this guide, familiarize yourself with the following:

- [Internal Application Load Balancer overview](https://docs.cloud.google.com/load-balancing/docs/l7-internal.md.txt), including the [Limitations](https://docs.cloud.google.com/load-balancing/docs/l7-internal.md.txt#limitations) section
- [VPC firewall rules overview](https://docs.cloud.google.com/vpc/docs/firewalls)

### Set up an SSL certificate resource

Create a Certificate Manager SSL certificate resource as described in
the following:

- [Deploy a global self-managed certificate](https://docs.cloud.google.com/certificate-manager/docs/deploy-self-managed.md.txt).
- [Create a Google-managed certificate issued by your Certificate Authority Service instance](https://docs.cloud.google.com/certificate-manager/docs/deploy-google-managed-cas.md.txt#create-google-managed-certificate-issued-by-your-cas-instance).
- [Create a Google-managed certificate with DNS authorization](https://docs.cloud.google.com/certificate-manager/docs/deploy-google-managed-dns-auth.md.txt#create_a_google-managed_certificate_with_dns_authorization).

We recommend using a Google-managed certificate.

### Permissions

To follow this guide, you must be able to create instances and modify a
network in a project. You must be either a project
[owner or editor](https://docs.cloud.google.com/iam/docs/roles-overview.md.txt#basic), or you must have all of
the following [Compute Engine IAM roles](https://docs.cloud.google.com/compute/docs/access/iam.md.txt).

| Task | Required role |
|---|---|
| Create networks, subnets, and load balancer components | [Compute Network Admin](https://docs.cloud.google.com/compute/docs/access/iam.md.txt#compute.networkAdmin) |
| Add and remove firewall rules | [Compute Security Admin](https://docs.cloud.google.com/compute/docs/access/iam.md.txt#compute.securityAdmin) |
| Create instances | [Compute Instance Admin](https://docs.cloud.google.com/compute/docs/access/iam.md.txt#compute.instanceAdmin.v1) |

For more information, see the following guides:

- [Roles and permissions](https://docs.cloud.google.com/load-balancing/docs/access-control.md.txt)
- [IAM Conditions for forwarding rules](https://docs.cloud.google.com/load-balancing/docs/access-control/iam-conditions.md.txt)

## Setup overview

You can configure load balancer as shown in the following diagram:
[![Cross-region internal Application Load Balancer high availability deployment.](https://docs.cloud.google.com/static/load-balancing/images/cross-reg-int-vm.svg)](https://docs.cloud.google.com/static/load-balancing/images/cross-reg-int-vm.svg) Cross-region internal Application Load Balancer high availability deployment (click to enlarge).

As shown in the diagram, this example creates a cross-region internal Application Load Balancer in a
VPC network, with one backend
service and two backend managed instance groups in the
`REGION_A` and `REGION_B` regions.

The diagram shows the following:

1. A VPC network with the following subnets:

   - Subnet `SUBNET_A` and a proxy-only subnet in `REGION_A`.
   - Subnet `SUBNET_B` and a proxy-only subnet in `REGION_B`.

   You must create [proxy-only subnets](https://docs.cloud.google.com/load-balancing/docs/proxy-only-subnets.md.txt)
   in each region of a VPC network where you use
   cross-region internal Application Load Balancers. The region's
   proxy-only subnet is shared among all cross-region internal Application Load Balancers in
   the region. Source addresses of packets sent from the load balancer
   to your service's backends are allocated from the
   proxy-only subnet.
   In this example, the proxy-only subnet for the region
   `REGION_A` has a primary IP address range of
   `10.129.0.0/23` and for `REGION_B`
   has a primary IP address range of `10.130.0.0/23` which is the
   recommended subnet size.
2. High availability setup has managed instance group backends for
   Compute Engine VM deployments in `REGION_A` and
   `REGION_B` regions.
   If backends in one region happen to be down, traffic fails over to the
   other region.

3. A global backend service that monitors the usage and health of
   backends.

4. A global URL map that parses the URL of a request and forwards
   requests to specific backend services based on the host and path of the
   request URL.

5. A global target HTTP or HTTPS proxy, receives a request from the
   user and forwards it to the URL map. For HTTPS, configure a global SSL
   certificate resource. The target proxy uses the SSL certificate to decrypt SSL
   traffic if you configure HTTPS load balancing. The target proxy can forward
   traffic to your instances by using HTTP or HTTPS.

6. Global forwarding rules, has the regional internal IP address of your
   load balancer, to forward each incoming request to the target proxy.

   The internal IP address associated with the forwarding rule can come from
   a subnet in the same network and region as the backends. Note the following
   conditions:
   - The IP address can (but does not need to) come from the same subnet as the backend instance groups.
   - The IP address must not come from a reserved proxy-only subnet that has its `--purpose` flag set to `GLOBAL_MANAGED_PROXY`.
   - If you want to [use the same internal IP address with multiple forwarding
     rules](https://docs.cloud.google.com/load-balancing/docs/l7-internal/setting-up-l7-internal#common_ip), set the IP address `--purpose` flag to `SHARED_LOADBALANCER_VIP`.
7. Optional: [Configure DNS routing policies](https://docs.cloud.google.com/load-balancing/docs/l7-internal/setting-up-l7-cross-reg-internal.md.txt#dns-policy-manager)
   of type `GEO` to route client traffic to the load balancer VIP in
   the region closest to the client.

## Configure the network and subnets

Within the VPC network, configure a subnet in each region
where your backends are configured. In addition, configure a `proxy-only-subnet`
in each region that you want to configure the load balancer.

This example uses the following VPC network, region, and
subnets:

- **Network** . The network is a [custom mode VPC
  network](https://docs.cloud.google.com/vpc/docs/vpc#subnet-ranges) named `NETWORK`.

- **Subnets for backends**.

  - A subnet named `SUBNET_A` in the `REGION_A` region uses `10.1.2.0/24` for its primary IP range.
  - A subnet named `SUBNET_B` in the `REGION_B` region uses `10.1.3.0/24` for its primary IP range.
- **Subnets for proxies**.

  - A subnet named `PROXY_SN_A` in the `REGION_A` region uses `10.129.0.0/23` for its primary IP range.
  - A subnet named `PROXY_SN_B` in the `REGION_B` region uses `10.130.0.0/23` for its primary IP range.

Cross-region load balancers can be accessed from any region within the VPC.
So clients from any region can globally access your load balancer backends.

> [!NOTE]
> **Note:** Subsequent steps in this guide use the network, region, and subnet parameters as outlined here.

### Configure the backend subnets

### Console

1. In the Google Cloud console, go to the **VPC networks** page.

   [Go to VPC networks](https://console.cloud.google.com/networking/networks/list)
2. Click **Create VPC network**.

3. Provide a **Name** for the network.

4. In the **Subnets** section, set the **Subnet creation mode** to
   **Custom**.

5. Create a subnet for the load balancer's backends. In the **New subnet**
   section, enter the following information:

   - Provide a **Name** for the subnet.
   - Select a **Region** : <var translate="no">REGION_A</var>
   - Enter an **IP address range** : `10.1.2.0/24`
6. Click **Done**.

7. Click **Add subnet**.

8. Create a subnet for the load balancer's backends. In the **New
   subnet** section, enter the following information:

   - Provide a **Name** for the subnet.
   - Select a **Region** : <var translate="no">REGION_B</var>
   - Enter an **IP address range** : `10.1.3.0/24`
9. Click **Done**.

10. Click **Create**.

### gcloud

1. Create the custom VPC network with the [`gcloud compute
   networks create` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/networks/create):

   ```
   gcloud compute networks create NETWORK \
       --subnet-mode=custom
   ```
2. Create a subnet in the `NETWORK` network in the
   `REGION_A` region with
   the [`gcloud compute networks subnets create` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/networks/subnets/create):

   ```
   gcloud compute networks subnets create SUBNET_A \
       --network=NETWORK \
       --range=10.1.2.0/24 \
       --region=REGION_A
   ```
3. Create a subnet in the `NETWORK` network in the
   `REGION_B` region with
   the [`gcloud compute networks subnets create` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/networks/subnets/create):

   ```
   gcloud compute networks subnets create SUBNET_B \
       --network=NETWORK \
       --range=10.1.3.0/24 \
       --region=REGION_B
   ```

### Terraform

To create the VPC network, use the [`google_compute_network` resource](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_network).

```terraform
resource "google_compute_network" "default" {
  auto_create_subnetworks = false
  name                    = "lb-network-crs-reg"
  provider                = google-beta
}
```

To create the VPC subnets in the `lb-network-crs-reg` network,
use the [`google_compute_subnetwork` resource](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_subnetwork).

```terraform
resource "google_compute_subnetwork" "subnet_a" {
  provider      = google-beta
  ip_cidr_range = "10.1.2.0/24"
  name          = "lbsubnet-uswest1"
  network       = google_compute_network.default.id
  region        = "us-west1"
}
```

```terraform
resource "google_compute_subnetwork" "subnet_b" {
  provider      = google-beta
  ip_cidr_range = "10.1.3.0/24"
  name          = "lbsubnet-useast1"
  network       = google_compute_network.default.id
  region        = "us-east1"
}
```

### API

Make a `POST` request to the
[`networks.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/networks/insert.md.txt).
Replace `PROJECT_ID` with your project ID.

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/networks

{
 "routingConfig": {
   "routingMode": "regional"
 },
 "name": "NETWORK",
 "autoCreateSubnetworks": false
}
```

Make a `POST` request to the
[`subnetworks.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/subnetworks/insert.md.txt).
Replace `PROJECT_ID` with your project ID.

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION_A/subnetworks

{
 "name": "SUBNET_A",
 "network": "projects/PROJECT_ID/global/networks/lb-network-crs-reg",
 "ipCidrRange": "10.1.2.0/24",
 "region": "projects/PROJECT_ID/regions/REGION_A",
}
```

Make a `POST` request to the
[`subnetworks.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/subnetworks/insert.md.txt).
Replace `PROJECT_ID` with your project ID.

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION_B/subnetworks

{
 "name": "SUBNET_B",
 "network": "projects/PROJECT_ID/global/networks/NETWORK",
 "ipCidrRange": "10.1.3.0/24",
 "region": "projects/PROJECT_ID/regions/REGION_B",
}
```

### Configure the proxy-only subnet

A [proxy-only subnet](https://docs.cloud.google.com/load-balancing/docs/proxy-only-subnets.md.txt) provides a
set of IP addresses that Google Cloud uses to run Envoy proxies on your behalf. The
proxies terminate connections from the client and create new connections to the
backends.

This proxy-only subnet is used by all Envoy-based regional load
balancers in the same region as the VPC network. There can only be
one active proxy-only subnet for a given purpose, per region, per network.

> [!IMPORTANT]
> **Important:** Don't try to assign addresses from the proxy-only subnet to your load balancer's forwarding rule or backends. You assign the forwarding rule's IP address and the backend instance IP addresses from [a different subnet range (or ranges)](https://docs.cloud.google.com/load-balancing/docs/l7-internal/setting-up-l7-cross-reg-internal.md.txt#create-network-subnets), not this one. Google Cloud reserves this subnet range for Google Cloud-managed proxies.

### Console

If you're using the Google Cloud console, you can wait and create the proxy-only
subnet later on the **Load balancing** page.

If you want to create the proxy-only subnet now, use the following steps:

1. In the Google Cloud console, go to the **VPC networks** page.

   [Go to VPC networks](https://console.cloud.google.com/networking/networks/list)
2. Click the name of the VPC network.
3. On the **Subnet** tab, click **Add subnet**.
4. Provide a **Name** for the proxy-only subnet.
5. Select a **Region** : <var translate="no">REGION_A</var>
6. In the **Purpose** list, select **Cross-region Managed Proxy**.
7. In the **IP address range** field, enter `10.129.0.0/23`.
8. Click **Add**.

Create the proxy-only subnet in `REGION_B`

1. On the **Subnet** tab, click **Add subnet**.
2. Provide a **Name** for the proxy-only subnet.
3. Select a **Region** : <var translate="no">REGION_B</var>
4. In the **Purpose** list, select **Cross-region Managed Proxy**.
5. In the **IP address range** field, enter `10.130.0.0/23`.
6. Click **Add**.

### gcloud

Create the proxy-only subnets with the [`gcloud compute networks subnets create` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/networks/subnets/create).

```
    gcloud compute networks subnets create PROXY_SN_A \
        --purpose=GLOBAL_MANAGED_PROXY \
        --role=ACTIVE \
        --region=REGION_A \
        --network=NETWORK \
        --range=10.129.0.0/23
    
```

```
    gcloud compute networks subnets create PROXY_SN_B \
        --purpose=GLOBAL_MANAGED_PROXY \
        --role=ACTIVE \
        --region=REGION_B \
        --network=NETWORK \
        --range=10.130.0.0/23
    
```

### Terraform

To create the VPC proxy-only subnet in the `lb-network-crs-reg`
network, use the [`google_compute_subnetwork` resource](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_subnetwork).

```terraform
resource "google_compute_subnetwork" "proxy_subnet_a" {
  provider      = google-beta
  ip_cidr_range = "10.129.0.0/23"
  name          = "proxy-only-subnet1"
  network       = google_compute_network.default.id
  purpose       = "GLOBAL_MANAGED_PROXY"
  region        = "us-west1"
  role          = "ACTIVE"
  lifecycle {
    ignore_changes = [ipv6_access_type]
  }
}
```

```terraform
resource "google_compute_subnetwork" "proxy_subnet_b" {
  provider      = google-beta
  ip_cidr_range = "10.130.0.0/23"
  name          = "proxy-only-subnet2"
  network       = google_compute_network.default.id
  purpose       = "GLOBAL_MANAGED_PROXY"
  region        = "us-east1"
  role          = "ACTIVE"
  lifecycle {
    ignore_changes = [ipv6_access_type]
  }
}
```

### API

Create the proxy-only subnets with the [`subnetworks.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/subnetworks/insert.md.txt), replacing
`PROJECT_ID` with your project ID.

```
    POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION_A/subnetworks

    {
      "name": " PROXY_SN_A",
      "ipCidrRange": "10.129.0.0/23",
      "network": "projects/PROJECT_ID/global/networks/NETWORK",
      "region": "projects/PROJECT_ID/regions/REGION_A",
      "purpose": "GLOBAL_MANAGED_PROXY",
      "role": "ACTIVE"
    }
   
```

```
    POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION_B/subnetworks

    {
      "name": "PROXY_SN_B",
      "ipCidrRange": "10.130.0.0/23",
      "network": "projects/PROJECT_ID/global/networks/NETWORK",
      "region": "projects/PROJECT_ID/regions/REGION_B",
      "purpose": "GLOBAL_MANAGED_PROXY",
      "role": "ACTIVE"
    }
   
```

### Configure firewall rules

This example uses the following firewall rules:

- **fw-ilb-to-backends** . An ingress rule, applicable to the instances being load
  balanced, that allows incoming SSH connectivity on TCP port `22` from any
  address. You can choose a more restrictive source IP address range for this rule; for
  example, you can specify just the IP address ranges of the system from which you
  initiate SSH sessions. This example uses the target tag `allow-ssh` to
  identify the VMs that the firewall rule applies to.

- **fw-healthcheck** . An ingress rule, applicable to the instances
  being load balanced, that allows all TCP traffic from the [Google Cloud
  health checking systems](https://docs.cloud.google.com/load-balancing/docs/health-check-concepts#ip-ranges).
  This example uses the target tag `load-balanced-backend` to identify the VMs
  that the firewall rule applies to.

- **fw-backends** . An ingress rule, applicable to the instances being
  load balanced, that allows TCP traffic on ports `80`, `443`, and `8080` from
  the internal Application Load Balancer's managed proxies. This example uses the target tag
  `load-balanced-backend` to identify the VMs that the firewall rule applies to.

Without these firewall rules, the [default deny
ingress](https://docs.cloud.google.com/vpc/docs/firewalls#default_firewall_rules) rule blocks incoming
traffic to the backend instances.

The [target tags](https://docs.cloud.google.com/vpc/docs/add-remove-network-tags.md.txt#targets_for_firewall_rules)
define the backend instances. Without the target tags, the firewall
rules apply to all of your backend instances in the VPC network.
When you create the backend VMs, make sure to
include the specified target tags, as shown in [Creating a managed instance
group](https://docs.cloud.google.com/load-balancing/docs/l7-internal/setting-up-l7-internal#creating_a_managed_instance_group).

### Console

1. In the Google Cloud console, go to the **Firewall policies** page.

   [Go to Firewall policies](https://console.cloud.google.com/net-security/firewall-manager/firewall-policies/list)
2. Click **Create firewall rule** to create the rule to allow incoming
   SSH connections:

   - **Name** : `fw-ilb-to-backends`
   - **Network** : <var translate="no">NETWORK</var>
   - **Direction of traffic** : **Ingress**
   - **Action on match** : **Allow**
   - **Targets** : **Specified target tags**
   - **Target tags** : `allow-ssh`
   - **Source filter** : **IPv4 ranges**
   - **Source IPv4 ranges** : `0.0.0.0/0`
   - **Protocols and ports** :
     - Choose **Specified protocols and ports**.
     - Select the **TCP** checkbox, and then enter `22` for the port number.
3. Click **Create**.

4. Click **Create firewall rule** a second time to create the rule to allow
   Google Cloud health checks:

   - **Name** : `fw-healthcheck`
   - **Network** : <var translate="no">NETWORK</var>
   - **Direction of traffic** : **Ingress**
   - **Action on match** : **Allow**
   - **Targets** : **Specified target tags**
   - **Target tags** : `load-balanced-backend`
   - **Source filter** : **IPv4 ranges**
   - **Source IPv4 ranges** : `35.191.0.0/16`
   - **Protocols and ports**:

     - Choose **Specified protocols and ports**.
     - Select the **TCP** checkbox, and then enter `80` for the port number.

     As a best practice, limit this rule to just the protocols and ports
     that match those used by your health check. If you use `tcp:80` for
     the protocol and port, Google Cloud can use
     HTTP on port `80` to contact your VMs, but it cannot use HTTPS on
     port `443` to contact them.
5. Click **Create**.

6. Click **Create firewall rule** a third time to create the rule to allow
   the load balancer's proxy servers to connect the backends:

   - **Name** : `fw-backends`
   - **Network** : <var translate="no">NETWORK</var>
   - **Direction of traffic** : **Ingress**
   - **Action on match** : **Allow**
   - **Targets** : **Specified target tags**
   - **Target tags** : `load-balanced-backend`
   - **Source filter** : **IPv4 ranges**
   - **Source IPv4 ranges** : `10.129.0.0/23` and `10.130.0.0/23`
   - **Protocols and ports** :
     - Choose **Specified protocols and ports**.
     - Select the **TCP** checkbox, and then enter `80, 443, 8080` for the port numbers.
7. Click **Create**.

### gcloud

1. Create the `fw-ilb-to-backends` firewall rule to allow SSH connectivity to
   VMs with the network tag `allow-ssh`. When you omit `source-ranges`,
   Google Cloud [interprets the rule to mean any
   source](https://docs.cloud.google.com/vpc/docs/firewalls#sources_for_the_rule).

   ```
   gcloud compute firewall-rules create fw-ilb-to-backends \
       --network=NETWORK \
       --action=allow \
       --direction=ingress \
       --target-tags=allow-ssh \
       --rules=tcp:22
   ```
2. Create the `fw-healthcheck` rule to allow Google Cloud
   health checks. This example allows all TCP traffic from health check
   probers; however, you can configure a narrower set of ports to meet your
   needs.

       gcloud compute firewall-rules create fw-healthcheck \
           --network=NETWORK \
           --action=allow \
           --direction=ingress \
           --source-ranges=35.191.0.0/16 \
           --target-tags=load-balanced-backend \
           --rules=tcp

3. Create the `fw-backends` rule to allow the internal Application Load Balancer's
   proxies to connect to your backends. Set `source-ranges` to the
   allocated ranges of your proxy-only subnet,
   for example, `10.129.0.0/23` and `10.130.0.0/23`.

   ```
   gcloud compute firewall-rules create fw-backends \
       --network=NETWORK \
       --action=allow \
       --direction=ingress \
       --source-ranges=source-range \
       --target-tags=load-balanced-backend \
       --rules=tcp:80,tcp:443,tcp:8080
   ```

### Terraform

To create the firewall rules, use the [`google_compute_firewall` resource](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_firewall).

```terraform
resource "google_compute_firewall" "fw_healthcheck" {
  name          = "gl7-ilb-fw-allow-hc"
  provider      = google-beta
  direction     = "INGRESS"
  network       = google_compute_network.default.id
  source_ranges = ["130.211.0.0/22", "35.191.0.0/16", "35.235.240.0/20"]
  allow {
    protocol = "tcp"
  }
}
```

```terraform
resource "google_compute_firewall" "fw_ilb_to_backends" {
  name          = "fw-ilb-to-fw"
  provider      = google-beta
  network       = google_compute_network.default.id
  source_ranges = ["0.0.0.0/0"]
  allow {
    protocol = "tcp"
    ports    = ["22", "80", "443", "8080"]
  }
}
```

```terraform
resource "google_compute_firewall" "fw_backends" {
  name          = "gl7-ilb-fw-allow-ilb-to-backends"
  direction     = "INGRESS"
  network       = google_compute_network.default.id
  source_ranges = ["10.129.0.0/23", "10.130.0.0/23"]
  target_tags   = ["http-server"]
  allow {
    protocol = "tcp"
    ports    = ["80", "443", "8080"]
  }
}
```

### API

Create the `fw-ilb-to-backends` firewall rule by making a `POST` request to
the [`firewalls.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/firewalls/insert.md.txt),
replacing `PROJECT_ID` with your project ID.

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/firewalls

{
 "name": "fw-ilb-to-backends",
 "network": "projects/PROJECT_ID/global/networks/NETWORK",
 "sourceRanges": [
   "0.0.0.0/0"
 ],
 "targetTags": [
   "allow-ssh"
 ],
 "allowed": [
  {
    "IPProtocol": "tcp",
    "ports": [
      "22"
    ]
  }
 ],
"direction": "INGRESS"
}
```

Create the `fw-healthcheck` firewall rule by making a `POST` request to
the [`firewalls.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/firewalls/insert.md.txt),
replacing `PROJECT_ID` with your project ID.

    POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/firewalls

    {
     "name": "fw-healthcheck",
     "network": "projects/PROJECT_ID/global/networks/NETWORK",
     "sourceRanges": [35.191.0.0/16],
     "targetTags": [
       "load-balanced-backend"
     ],
     "allowed": [
       {
         "IPProtocol": "tcp"
       }
     ],
     "direction": "INGRESS"
    }

Create the `fw-backends` firewall rule to allow TCP traffic within the
proxy subnet for the
[`firewalls.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/firewalls/insert.md.txt),
replacing `PROJECT_ID` with your project ID.

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/firewalls

{
 "name": "fw-backends",
 "network": "projects/PROJECT_ID/global/networks/NETWORK",
 "sourceRanges": [
   "10.129.0.0/23",
   "10.130.0.0/23"
 ],
 "targetTags": [
   "load-balanced-backend"
 ],
 "allowed": [
   {
     "IPProtocol": "tcp",
     "ports": [
       "80"
     ]
   },
 {
     "IPProtocol": "tcp",
     "ports": [
       "443"
     ]
   },
   {
     "IPProtocol": "tcp",
     "ports": [
       "8080"
     ]
   }
 ],
 "direction": "INGRESS"
}
```

## Create a managed instance group

This section shows how to create a template and a managed instance group. The
managed instance group provides VM instances running the backend servers of an
example cross-region internal Application Load Balancer. For your instance group, you can define an HTTP
service and map a port name to the relevant port. The backend service
of the load balancer forwards traffic to the [named ports](https://docs.cloud.google.com/load-balancing/docs/backend-service.md.txt#named_ports).
Traffic from clients is load balanced to backend servers. For demonstration
purposes, backends serve their own hostnames.

### Console

1. In the Google Cloud console, go to
   the **Instance templates** page.

   [Go to Instance templates](https://console.cloud.google.com/compute/instanceTemplates/list)
   1. Click **Create instance template**.
   2. For **Name** , enter `gil7-backendeast1-template`.
   3. Ensure that the **Boot disk** is set to a Debian image, such as **Debian GNU/Linux 12 (bookworm)** . These instructions use commands that are only available on Debian, such as `apt-get`.
   4. Click **Advanced options**.
   5. Click **Networking** and configure the following fields:
      1. For **Network tags** , enter `allow-ssh` and `load-balanced-backend`.
      2. For **Network interfaces** , select the following:
         - **Network** : <var translate="no">NETWORK</var>
         - **Subnet** : <var translate="no">SUBNET_B</var>
   6. Click **Management** . Enter the following script into the
      **Startup script** field.

      ```
      #! /bin/bash
      apt-get update
      apt-get install apache2 -y
      a2ensite default-ssl
      a2enmod ssl
      vm_hostname="$(curl -H "Metadata-Flavor:Google" \
      http://169.254.169.254/computeMetadata/v1/instance/name)"
      echo "Page served from: $vm_hostname" | \
      tee /var/www/html/index.html
      systemctl restart apache2
      ```
   7. Click **Create**.

   8. Click **Create instance template**.

   9. For **Name** , enter `gil7-backendwest1-template`.

   10. Ensure that the **Boot disk** is set to a Debian image, such as
       **Debian GNU/Linux 12 (bookworm)** . These instructions use commands that
       are only available on Debian, such as `apt-get`.

   11. Click **Advanced options**.

   12. Click **Networking** and configure the following fields:

       1. For **Network tags** , enter `allow-ssh` and `load-balanced-backend`.
       2. For **Network interfaces** , select the following:
          - **Network** : <var translate="no">NETWORK</var>
          - **Subnet** : <var translate="no">SUBNET_A</var>
   13. Click **Management** . Enter the following script into the
       **Startup script** field.

       ```
       #! /bin/bash
       apt-get update
       apt-get install apache2 -y
       a2ensite default-ssl
       a2enmod ssl
       vm_hostname="$(curl -H "Metadata-Flavor:Google" \
       http://169.254.169.254/computeMetadata/v1/instance/name)"
       echo "Page served from: $vm_hostname" | \
       tee /var/www/html/index.html
       systemctl restart apache2
       ```
   14. Click **Create**.

2. In the Google Cloud console, go to the **Instance groups** page.

   [Go to Instance groups](https://console.cloud.google.com/compute/instanceGroups)
   1. Click **Create instance group**.
   2. Select **New managed instance group (stateless)** . For more information, see [Stateless or stateful MIGs](https://docs.cloud.google.com/compute/docs/instance-groups/creating-groups-of-managed-instances.md.txt#stateless_or_stateful_migs).
   3. For **Name** , enter `gl7-ilb-mig-a`.
   4. For **Location** , select **Single zone**.
   5. For **Region** , select <var translate="no">REGION_A</var>.
   6. For **Zone** , select <var translate="no">ZONE_A</var>.
   7. For **Instance template** , select `gil7-backendwest1-template`.
   8. Specify the number of instances that you want to create in the group.

      For this example, specify the following options under **Autoscaling**:
      - For **Autoscaling mode** , select `Off:do not autoscale`.
      - For **Maximum number of instances** , enter `2`.

      Optionally, in the **Autoscaling** section of the UI, you can configure
      the instance group to [automatically add or remove
      instances](https://docs.cloud.google.com/compute/docs/autoscaler) based on instance CPU use.
   9. Click **Create**.

   10. Click **Create instance group**.

   11. Select **New managed instance group (stateless)** . For more
       information, see [Stateless or stateful MIGs](https://docs.cloud.google.com/compute/docs/instance-groups/creating-groups-of-managed-instances.md.txt#stateless_or_stateful_migs).

   12. For **Name** , enter `gl7-ilb-mig-b`.

   13. For **Location** , select **Single zone**.

   14. For **Region** , select <var translate="no">REGION_B</var>.

   15. For **Zone** , select <var translate="no">ZONE_B</var>.

   16. For **Instance template** , select `gil7-backendeast1-template`.

   17. Specify the number of instances that you want to create in the group.

       For this example, specify the following options under **Autoscaling**:
       - For **Autoscaling mode** , select `Off:do not autoscale`.
       - For **Maximum number of instances** , enter `2`.

       Optionally, in the **Autoscaling** section of the UI, you can configure
       the instance group to [automatically add or remove
       instances](https://docs.cloud.google.com/compute/docs/autoscaler) based on instance CPU usage.
   18. Click **Create**.

### gcloud

The gcloud CLI instructions in this guide assume that you are using
[Cloud Shell](https://docs.cloud.google.com/shell/docs.md.txt) or another environment with bash installed.

1. Create a VM instance template with HTTP server with the
   [`gcloud compute instance-templates create` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/instance-templates/create.md.txt).

   ```
   gcloud compute instance-templates create gil7-backendwest1-template \
     --region=REGION_A \
     --network=NETWORK \
     --subnet=SUBNET_A \
     --tags=allow-ssh,load-balanced-backend \
     --image-family=debian-12 \
     --image-project=debian-cloud \
     --metadata=startup-script='#! /bin/bash
       apt-get update
       apt-get install apache2 -y
       a2ensite default-ssl
       a2enmod ssl
       vm_hostname="$(curl -H "Metadata-Flavor:Google" \
       http://169.254.169.254/computeMetadata/v1/instance/name)"
       echo "Page served from: $vm_hostname" | \
       tee /var/www/html/index.html
       systemctl restart apache2'
   ```

   ```
   gcloud compute instance-templates create gil7-backendeast1-template \
       --region=REGION_B \
       --network=NETWORK \
       --subnet=SUBNET_B \
       --tags=allow-ssh,load-balanced-backend \
       --image-family=debian-12 \
       --image-project=debian-cloud \
       --metadata=startup-script='#! /bin/bash
         apt-get update
         apt-get install apache2 -y
         a2ensite default-ssl
         a2enmod ssl
         vm_hostname="$(curl -H "Metadata-Flavor:Google" \
         http://169.254.169.254/computeMetadata/v1/instance/name)"
         echo "Page served from: $vm_hostname" | \
         tee /var/www/html/index.html
         systemctl restart apache2'
   ```
2. Create a managed instance group in the zone with the [`gcloud compute
   instance-groups managed create` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/instance-groups/managed/create).

   ```
   gcloud compute instance-groups managed create gl7-ilb-mig-a \
       --zone=ZONE_A \
       --size=2 \
       --template=gil7-backendwest1-template
   ```

   ```
   gcloud compute instance-groups managed create gl7-ilb-mig-b \
       --zone=ZONE_B \
       --size=2 \
       --template=gil7-backendeast1-template
   ```

### Terraform

To create the instance template, use the [`google_compute_instance_template` resource](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_instance_template).

```terraform
resource "google_compute_instance_template" "instance_template_a" {
  name         = "gil7-backendwest1-template"
  provider     = google-beta
  machine_type = "e2-small"
  region       = "us-west1"
  tags         = ["http-server"]

  network_interface {
    network    = google_compute_network.default.id
    subnetwork = google_compute_subnetwork.subnet_a.id
    access_config {
      # add external ip to fetch packages
    }
  }
  disk {
    source_image = "debian-cloud/debian-13"
    auto_delete  = true
    boot         = true
  }

  # install nginx and serve a simple web page
  metadata = {
    startup-script = <<-EOF1
      #! /bin/bash
      set -euo pipefail

      export DEBIAN_FRONTEND=noninteractive
      apt-get update
      apt-get install -y nginx-light jq

      NAME=$(curl -H "Metadata-Flavor: Google" "http://metadata.google.internal/computeMetadata/v1/instance/hostname")
      IP=$(curl -H "Metadata-Flavor: Google" "http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip")
      METADATA=$(curl -f -H "Metadata-Flavor: Google" "http://metadata.google.internal/computeMetadata/v1/instance/attributes/?recursive=True" | jq 'del(.["startup-script"])')

      cat <<EOF > /var/www/html/index.html
      <pre>
      Name: $NAME
      IP: $IP
      Metadata: $METADATA
      </pre>
      EOF
    EOF1
  }
  lifecycle {
    create_before_destroy = true
  }
}
```

```terraform
resource "google_compute_instance_template" "instance_template_b" {
  name         = "gil7-backendeast1-template"
  provider     = google-beta
  machine_type = "e2-small"
  region       = "us-east1"
  tags         = ["http-server"]

  network_interface {
    network    = google_compute_network.default.id
    subnetwork = google_compute_subnetwork.subnet_b.id
    access_config {
      # add external ip to fetch packages
    }
  }
  disk {
    source_image = "debian-cloud/debian-13"
    auto_delete  = true
    boot         = true
  }

  # install nginx and serve a simple web page
  metadata = {
    startup-script = <<-EOF1
      #! /bin/bash
      set -euo pipefail

      export DEBIAN_FRONTEND=noninteractive
      apt-get update
      apt-get install -y nginx-light jq

      NAME=$(curl -H "Metadata-Flavor: Google" "http://metadata.google.internal/computeMetadata/v1/instance/hostname")
      IP=$(curl -H "Metadata-Flavor: Google" "http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip")
      METADATA=$(curl -f -H "Metadata-Flavor: Google" "http://metadata.google.internal/computeMetadata/v1/instance/attributes/?recursive=True" | jq 'del(.["startup-script"])')

      cat <<EOF > /var/www/html/index.html
      <pre>
      Name: $NAME
      IP: $IP
      Metadata: $METADATA
      </pre>
      EOF
    EOF1
  }
  lifecycle {
    create_before_destroy = true
  }
}
```

To create the managed instance group, use the [`google_compute_instance_group_manager` resource](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_instance_group_manager).

```terraform
resource "google_compute_region_instance_group_manager" "mig_a" {
  name     = "gl7-ilb-miga"
  provider = google-beta
  region   = "us-west1"
  version {
    instance_template = google_compute_instance_template.instance_template_a.id
    name              = "primary"
  }
  base_instance_name = "vm"
  target_size        = 2
}
```

```terraform
resource "google_compute_region_instance_group_manager" "mig_b" {
  name     = "gl7-ilb-migb"
  provider = google-beta
  region   = "us-east1"
  version {
    instance_template = google_compute_instance_template.instance_template_b.id
    name              = "primary"
  }
  base_instance_name = "vm"
  target_size        = 2
}
```

### API

Create the instance template with the
[`instanceTemplates.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/instanceTemplates/insert.md.txt),
replacing `PROJECT_ID` with your project ID.

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/instanceTemplates

{
  "name":"gil7-backendwest1-template",
  "properties":{
     "machineType":"e2-standard-2",
     "tags":{
       "items":[
         "allow-ssh",
         "load-balanced-backend"
       ]
     },
     "metadata":{
        "kind":"compute#metadata",
        "items":[
          {
            "key":"startup-script",
            "value":"#! /bin/bash\napt-get update\napt-get install
            apache2 -y\na2ensite default-ssl\na2enmod ssl\n
            vm_hostname=\"$(curl -H \"Metadata-Flavor:Google\"
            \\\nhttp://169.254.169.254/computeMetadata/v1/instance/name)\"\n
            echo \"Page served from: $vm_hostname\" | \\\ntee
            /var/www/html/index.html\nsystemctl restart apache2"
          }
        ]
     },
     "networkInterfaces":[
       {
         "network":"projects/PROJECT_ID/global/networks/NETWORK",
         "subnetwork":"regions/REGION_A/subnetworks/SUBNET_A",
         "accessConfigs":[
           {
             "type":"ONE_TO_ONE_NAT"
           }
         ]
       }
     ],
     "disks":[
       {
         "index":0,
         "boot":true,
         "initializeParams":{
           "sourceImage":"projects/debian-cloud/global/images/family/debian-12"
         },
         "autoDelete":true
       }
     ]
  }
}
```

Create a managed instance group in each zone with the
[`instanceGroupManagers.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/instanceGroupManagers/insert.md.txt),
replacing `PROJECT_ID` with your project ID.

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/{zone}/instanceGroupManagers

{
  "name": "gl7-ilb-mig-a",
  "zone": "projects/PROJECT_ID/zones/ZONE_A",
  "instanceTemplate": "projects/PROJECT_ID/global/instanceTemplates/gil7-backendwest1-template",
  "baseInstanceName": "gl7-ilb-mig-b",
  "targetSize": 2
}
```

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/instanceTemplates

{
  "name":"gil7-backendeast1-template",
  "properties":{
     "machineType":"e2-standard-2",
     "tags":{
       "items":[
         "allow-ssh",
         "load-balanced-backend"
       ]
     },
     "metadata":{
        "kind":"compute#metadata",
        "items":[
          {
            "key":"startup-script",
            "value":"#! /bin/bash\napt-get update\napt-get install
            apache2 -y\na2ensite default-ssl\na2enmod ssl\n
            vm_hostname=\"$(curl -H \"Metadata-Flavor:Google\"
            \\\nhttp://169.254.169.254/computeMetadata/v1/instance/name)\"\n
            echo \"Page served from: $vm_hostname\" | \\\ntee
            /var/www/html/index.html\nsystemctl restart apache2"
          }
        ]
     },
     "networkInterfaces":[
       {
         "network":"projects/PROJECT_ID/global/networks/NETWORK",
         "subnetwork":"regions/REGION_B/subnetworks/SUBNET_B",
         "accessConfigs":[
           {
             "type":"ONE_TO_ONE_NAT"
           }
         ]
       }
     ],
     "disks":[
       {
         "index":0,
         "boot":true,
         "initializeParams":{
           "sourceImage":"projects/debian-cloud/global/images/family/debian-12"
         },
         "autoDelete":true
       }
     ]
  }
}
```

Create a managed instance group in each zone with the
[`instanceGroupManagers.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/instanceGroupManagers/insert.md.txt),
replacing `PROJECT_ID` with your project ID.

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/{zone}/instanceGroupManagers

{
  "name": "gl7-ilb-mig-b",
  "zone": "projects/PROJECT_ID/zones/ZONE_B",
  "instanceTemplate": "projects/PROJECT_ID/global/instanceTemplates/gil7-backendwest1-template",
  "baseInstanceName": "gl7-ilb-mig-b",
  "targetSize": 2
}
```

## Configure the load balancer

This example shows you how to create the following cross-region internal Application Load Balancer
resources:

- A global HTTP health check.
- A global backend service with the managed instance groups as the backend.
- A [URL map](https://docs.cloud.google.com/load-balancing/docs/l7-internal/traffic-management.md.txt). Make sure to refer to a global URL map for the target HTTP(S) proxy. A global URL map routes requests to a global backend service based on rules that you define for the host and path of an incoming URL. A global URL map can be referenced by a global target proxy rule.
- A global SSL certificate (for HTTPS).
- A global target proxy.
- Two global forwarding rules with regional IP addresses.
  For the forwarding rule's IP address, use the `SUBNET_A`
  or `SUBNET_B` IP address range. If you
  try to use the [proxy-only subnet](https://docs.cloud.google.com/load-balancing/docs/proxy-only-subnets.md.txt),
  forwarding rule creation fails.

### Console


#### Select the load balancer type

1. In the Google Cloud console, go to the **Load balancing** page.

   [Go to Load balancing](https://console.cloud.google.com/net-services/loadbalancing/list)
2. Click **Create load balancer**.
3. For **Type of load balancer** , select **Application Load Balancer
   (HTTP/HTTPS)** and click **Next**.
4. For **Public facing or internal** , select **Internal** and click **Next**.
5. For **Cross-region or single region deployment** , select **Best for
   cross-region workloads** and click **Next**.
6. Click **Configure**.

**Basic configuration**

1. Provide a **Name** for the load balancer.
2. For **Network** , select <var translate="no">NETWORK</var>.

**Configure the frontend with two forwarding rules**

For HTTP:

1. Click **Frontend configuration**.
   1. Provide a **Name** for the forwarding rule.
   2. In the **Subnetwork region** list, select <var translate="no">REGION_A</var>.   

      **Reserve a proxy-only subnet**

   > [!NOTE]
   > If you already configured the proxy-only subnet, the **Reserve subnet** button isn't displayed. You can continue with the next steps.

   3. In the **Subnetwork** list, select <var translate="no">SUBNET_A</var>.
   4. In the **IP address** list, click **Create IP address** . The **Reserve a static internal IP address** page opens.
      - Provide a **Name** for the static IP address.
      - In the **Static IP address** list, select **Let me choose**.
      - In the **Custom IP address** field, enter `10.1.2.99`.
      - Select **Reserve**.
2. Click **Done**.
3. To add the second forwarding rule, click **Add frontend IP and port**.
   1. Provide a **Name** for the forwarding rule.
   2. In the **Subnetwork region** list, select <var translate="no">REGION_B</var>.   

      **Reserve a proxy-only subnet**

   > [!NOTE]
   > Because you have already configured the [proxy-only subnet](https://docs.cloud.google.com/load-balancing/docs/l7-internal/setting-up-l7-cross-reg-internal.md.txt#create-proxy-only-subnet), the **Reserve subnet** button isn't displayed. You can continue with the next steps.

   3. In the **Subnetwork** list, select <var translate="no">SUBNET_B</var>.
   4. In the **IP address** list, click **Create IP address** . The **Reserve a static internal IP address** page opens.
      - Provide a **Name** for the static IP address.
      - In the **Static IP address** list, select **Let me choose**.
      - In the **Custom IP address** field, enter `10.1.3.99`.
      - Select **Reserve**.
4. Click **Done**.

For HTTPS:

To assign an SSL certificate to the target HTTPS proxy of the load
balancer, you need to use a Certificate Manager
certificate.

1. Click **Frontend configuration**.
   1. Provide a **Name** for the forwarding rule.
   2. In the **Protocol** field, select `HTTPS (includes HTTP/2)`.
   3. Ensure that the **Port** is set to `443`.
   4. In the **Subnetwork region** list, select <var translate="no">REGION_A</var>.   

      **Reserve a proxy-only subnet**

   > [!NOTE]
   > Because you have already configured the [proxy-only subnet](https://docs.cloud.google.com/load-balancing/docs/l7-internal/setting-up-l7-cross-reg-internal.md.txt#create-proxy-only-subnet), the **Reserve subnet** button isn't displayed. You can continue with the next steps.

   5. In the **Subnetwork** list, select <var translate="no">SUBNET_A</var>.
   6. In the **IP address** list, click **Create IP address** . The **Reserve a static internal IP address** page opens.
      - Provide a **Name** for the static IP address.
      - In the **Static IP address** list, select **Let me choose**.
      - In the **Custom IP address** field, enter `10.1.3.99`.
      - Select **Reserve**.
   7. Click **Add certificate** to select an existing certificate or create a new certificate.

      If you already have an existing Certificate Manager certificate to select, do the following:
      1. Click **Add Certificate**.
      2. Click **Select an existing certificate** and select the certificate from the list of certificates.
      3. Click **Select**.

      After you select the new Certificate Manager certificate, it appears in the list of certificates.

      To create a new Certificate Manager certificate, do the following:
      1. Click **Add Certificate**.
      2. Click **Create a new certificate**.
      3. To create a new certificate, follow the steps starting from *step 3* as outlined in any one of the following configuration methods in the Certificate Manager documentation:
         - [Create a Google-managed certificate referencing the DNS authorization](https://docs.cloud.google.com/certificate-manager/docs/deploy-google-managed-dns-auth-cross-reg.md.txt#create-google-managed-certificate-dns-auth)
         - [Create a Google-managed certificate issued by your CA Service instance](https://docs.cloud.google.com/certificate-manager/docs/deploy-google-managed-cas-cross-reg.md.txt#create-google-managed-certificate-issued-by-your-cas-instance)
         - [Upload a self-managed certificate to Certificate Manager](https://docs.cloud.google.com/certificate-manager/docs/deploy-self-managed-cross-reg.md.txt#upload-self-managed)
   8. Select an SSL policy from the **SSL policy** list. If you have not created any SSL policies, a [default Google Cloud SSL policy](https://docs.cloud.google.com/load-balancing/docs/ssl-policies-concepts.md.txt#default-ssl-policy) is applied.
   9. Click **Done**.
2. **Add the second frontend configuration:**
   1. Provide a **Name** for the frontend configuration.
   2. In the **Protocol** field, select `HTTPS (includes HTTP/2)`.
   3. Ensure that the **Port** is set to `443`.
   4. In the **Subnetwork region** list, select <var translate="no">REGION_B</var>.   

      **Reserve a proxy-only subnet**

   > [!NOTE]
   > Because you have already configured the [proxy-only subnet](https://docs.cloud.google.com/load-balancing/docs/l7-internal/setting-up-l7-cross-reg-internal.md.txt#create-proxy-only-subnet), the **Reserve subnet** button isn't displayed. You can continue with the next steps.

   5. In the **Subnetwork** list, select <var translate="no">SUBNET_B</var>.
   6. In the **IP address** list, click **Create IP address** . The **Reserve a static internal IP address** page opens.
      - Provide a **Name** for the static IP address.
      - In the **Static IP address** list, select **Let me choose**.
      - In the **Custom IP address** field, enter `10.1.3.99`.
      - Select **Reserve**.
   7. Click **Add certificate** and then select an existing certificate or create a new certificate.
   8. Select an SSL policy from the **SSL policy** list. If you have not created any SSL policies, a [default Google Cloud SSL policy](https://docs.cloud.google.com/load-balancing/docs/ssl-policies-concepts.md.txt#default-ssl-policy) is applied.
   9. Click **Done**.
**Configure the backend service**
   1. Click **Backend configuration**.
   2. In the **Create or select backend services** list, click **Create a
      backend service**.
   3. Provide a **Name** for the backend service.
   4. For **Protocol** , select **HTTP**.
   5. For **Named Port** , enter `http`.
   6. In the **Backend type** list, select **Instance group**.
   7. In the **Health check** list, click **Create a health check**, and then enter the following information:
      - In the **Name** field, enter `global-http-health-check`.
      - In the **Protocol** list, select `HTTP`.
      - In the **Port** field, enter `80`.
      - Click **Create**.
   8. In the **New backend** section:
      1. In the **Instance group** list, select `gl4-ilb-miga` in <var translate="no">REGION_A</var>.
      2. Set **Port numbers** to `80`.
      3. For the **Balancing mode** , select **Utilization**.
      4. Click **Done**.
      5. To add another backend, click **Add backend**.
      6. In the **Instance group** list, select `gl4-ilb-migb` in <var translate="no">REGION_B</var>.
      7. Set **Port numbers** to `80`.
      8. Click **Done**.
3. **Configure the routing rules**
   1. Click **Routing rules**.
   2. For **Mode** , select **Simple host and path rule**.
   3. Ensure that there is only one backend service for any unmatched host and any unmatched path.
4. **Review the configuration**
   1. Click **Review and finalize**.
   2. Review your load balancer configuration settings.
   3. Click **Create**.

<br />

### gcloud

1. Define the HTTP health check with the [`gcloud compute health-checks
   create http` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/health-checks/create/http).

   ```
   gcloud compute health-checks create http global-http-health-check \
      --use-serving-port \
      --global
   ```
2. Define the backend service with the [`gcloud compute backend-services
   create` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/backend-services/create).

   ```
   gcloud compute backend-services create BACKEND_SERVICE_NAME \
     --load-balancing-scheme=INTERNAL_MANAGED \
     --protocol=HTTP \
     --enable-logging \
     --logging-sample-rate=1.0 \
     --health-checks=global-http-health-check \
     --global-health-checks \
     --global
   ```
3. Add backends to the backend service with the [`gcloud compute backend-services
   add-backend` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/backend-services/add-backend).

   ```
   gcloud compute backend-services add-backend BACKEND_SERVICE_NAME \
     --balancing-mode=UTILIZATION \
     --instance-group=gl7-ilb-mig-a \
     --instance-group-zone=ZONE_A \
     --global
   ```

   ```
   gcloud compute backend-services add-backend BACKEND_SERVICE_NAME \
     --balancing-mode=UTILIZATION \
     --instance-group=gl7-ilb-mig-b \
     --instance-group-zone=ZONE_B \
     --global
   ```
4. Create the URL map with the [`gcloud compute url-maps
   create` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/url-maps/create).

   ```
   gcloud compute url-maps create gl7-gilb-url-map \
     --default-service=BACKEND_SERVICE_NAME \
     --global
   ```
5. Create the target proxy.

   **For HTTP:**

   Create the target proxy with the [`gcloud compute target-http-proxies
   create` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/target-http-proxies/create).

   ```
   gcloud compute target-http-proxies create gil7-http-proxy \
     --url-map=gl7-gilb-url-map \
     --global
   ```

   **For HTTPS:**

   To create a Google-managed certificate, see the following documentation:
   - [Create a Google-managed certificate issued by your Certificate Authority Service instance](https://docs.cloud.google.com/certificate-manager/docs/deploy-google-managed-cas.md.txt#create-google-managed-certificate-issued-by-your-cas-instance).
   - [Create a Google-managed certificate with DNS authorization](https://docs.cloud.google.com/certificate-manager/docs/deploy-google-managed-dns-auth.md.txt#create_a_google-managed_certificate_with_dns_authorization).

   After you create the Google-managed certificate,
   [attach the certificate directly to the target proxy](https://docs.cloud.google.com/certificate-manager/docs/deploy-google-managed-dns-auth.md.txt#att-cert-target-proxy).
   Certificate maps are not supported by cross-region internal Application Load Balancers.

   To create a self-managed certificate, see the following documentation:
   - [Create a private key and certificate](https://docs.cloud.google.com/load-balancing/docs/ssl-certificates/self-managed-certs.md.txt#create-key-and-cert).

   Assign your file paths to variable names.

   ```
   export LB_CERT=PATH_TO_PEM_FORMATTED_FILE
   ```

   ```
   export LB_PRIVATE_KEY=PATH_TO_LB_PRIVATE_KEY_FILE
   ```

   Create an all region SSL certificate using the [`gcloud beta certificate-manager
   certificates create` command](https://docs.cloud.google.com/sdk/gcloud/reference/beta/certificate-manager/certificates/create).

   ```
   gcloud certificate-manager certificates create gilb-certificate \
     --private-key-file=$LB_PRIVATE_KEY \
     --certificate-file=$LB_CERT \
     --scope=all-regions
   ```

   Use the SSL certificate to create a target proxy with the [`gcloud
   compute target-https-proxies create` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/target-https-proxies/create)

   ```
   gcloud compute target-https-proxies create gil7-https-proxy \
     --url-map=gl7-gilb-url-map \
     --certificate-manager-certificates=gilb-certificate \
     --global
   ```
6. Create two forwarding rules, one with a VIP (`10.1.2.99`) in the
   `REGION_B` region and another one with a
   VIP (`10.1.3.99`) in the `REGION_A` region.
   For more information, see
   [Reserve a static internal IPv4 address](https://docs.cloud.google.com/compute/docs/ip-addresses/reserve-static-internal-ip-address#reservenewip).

   For custom networks, you must reference the subnet in the forwarding
   rule. Note that this is the VM subnet, not the proxy subnet.

   **For HTTP:**

   Use the [`gcloud compute forwarding-rules
   create` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/forwarding-rules/create)
   with the correct flags.

   ```
   gcloud compute forwarding-rules create FWRULE_A \
     --load-balancing-scheme=INTERNAL_MANAGED \
     --network=NETWORK \
     --subnet=SUBNET_A \
     --subnet-region=REGION_A \
     --address=10.1.2.99 \
     --ports=80 \
     --target-http-proxy=gil7-http-proxy \
     --global
   ```

   ```
   gcloud compute forwarding-rules create FWRULE_B \
     --load-balancing-scheme=INTERNAL_MANAGED \
     --network=NETWORK \
     --subnet=SUBNET_B \
     --subnet-region=REGION_B \
     --address=10.1.3.99 \
     --ports=80 \
     --target-http-proxy=gil7-http-proxy \
     --global
   ```

   **For HTTPS:**

   Use the [`gcloud compute forwarding-rules
   create` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/forwarding-rules/create)
   with the correct flags.

   ```
   gcloud compute forwarding-rules create FWRULE_A \
     --load-balancing-scheme=INTERNAL_MANAGED \
     --network=NETWORK \
     --subnet=SUBNET_A \
     --subnet-region=REGION_A \
     --address=10.1.2.99 \
     --ports=443 \
     --target-https-proxy=gil7-https-proxy \
     --global
   ```

   ```
   gcloud compute forwarding-rules create FWRULE_B \
     --load-balancing-scheme=INTERNAL_MANAGED \
     --network=NETWORK \
     --subnet=SUBNET_B \
     --subnet-region=REGION_B \
     --address=10.1.3.99 \
     --ports=443 \
     --target-https-proxy=gil7-https-proxy \
     --global
   ```

### Terraform

To create the health check, use the [`google_compute_health_check` resource](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_health_check).

```terraform
resource "google_compute_health_check" "default" {
  provider = google-beta
  name     = "global-http-health-check"
  http_health_check {
    port_specification = "USE_SERVING_PORT"
  }
}
```

To create the backend service, use the [`google_compute_backend_service` resource](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_backend_service).

```terraform
resource "google_compute_backend_service" "default" {
  name                  = "gl7-gilb-backend-service"
  provider              = google-beta
  protocol              = "HTTP"
  load_balancing_scheme = "INTERNAL_MANAGED"
  timeout_sec           = 10
  health_checks         = [google_compute_health_check.default.id]
  backend {
    group           = google_compute_region_instance_group_manager.mig_a.instance_group
    balancing_mode  = "UTILIZATION"
    capacity_scaler = 1.0
  }
  backend {
    group           = google_compute_region_instance_group_manager.mig_b.instance_group
    balancing_mode  = "UTILIZATION"
    capacity_scaler = 1.0
  }
}
```

To create the URL map, use the [`google_compute_url_map` resource](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_url_map).

```terraform
resource "google_compute_url_map" "default" {
  name            = "gl7-gilb-url-map"
  provider        = google-beta
  default_service = google_compute_backend_service.default.id
}
```

To create the target HTTP proxy, use the [`google_compute_target_http_proxy` resource](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_target_http_proxy).

```terraform
resource "google_compute_target_http_proxy" "default" {
  name     = "gil7target-http-proxy"
  provider = google-beta
  url_map  = google_compute_url_map.default.id
}
```

To create the forwarding rules, use the [`google_compute_forwarding_rule` resource](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_forwarding_rule).

```terraform
resource "google_compute_global_forwarding_rule" "fwd_rule_a" {
  provider              = google-beta
  depends_on            = [google_compute_subnetwork.proxy_subnet_a]
  ip_address            = "10.1.2.99"
  ip_protocol           = "TCP"
  load_balancing_scheme = "INTERNAL_MANAGED"
  name                  = "gil7forwarding-rule-a"
  network               = google_compute_network.default.id
  port_range            = "80"
  target                = google_compute_target_http_proxy.default.id
  subnetwork            = google_compute_subnetwork.subnet_a.id
}
```

```terraform
resource "google_compute_global_forwarding_rule" "fwd_rule_b" {
  provider              = google-beta
  depends_on            = [google_compute_subnetwork.proxy_subnet_b]
  ip_address            = "10.1.3.99"
  ip_protocol           = "TCP"
  load_balancing_scheme = "INTERNAL_MANAGED"
  name                  = "gil7forwarding-rule-b"
  network               = google_compute_network.default.id
  port_range            = "80"
  target                = google_compute_target_http_proxy.default.id
  subnetwork            = google_compute_subnetwork.subnet_b.id
}
```

To learn how to apply or remove a Terraform configuration, see
[Basic Terraform commands](https://docs.cloud.google.com/docs/terraform/basic-commands.md.txt).

### API

Create the health check by making a `POST` request to the
[`healthChecks.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/healthChecks/insert.md.txt),
replacing `PROJECT_ID` with your project ID.

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/healthChecks

{
"name": "global-http-health-check",
"type": "HTTP",
"httpHealthCheck": {
  "portSpecification": "USE_SERVING_PORT"
}
}
```

Create the global backend service by making a `POST` request to the
[`backendServices.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/backendServices/insert.md.txt),
replacing `PROJECT_ID` with your project ID.

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendServices

{
"name": "BACKEND_SERVICE_NAME",
"backends": [
  {
    "group": "projects/PROJECT_ID/zones/ZONE_A/instanceGroups/gl7-ilb-mig-a",
    "balancingMode": "UTILIZATION"
  },
  {
    "group": "projects/PROJECT_ID/zones/ZONE_B/instanceGroups/gl7-ilb-mig-b",
    "balancingMode": "UTILIZATION"
  }
],
"healthChecks": [
  "projects/PROJECT_ID/regions/global/healthChecks/global-http-health-check"
],
"loadBalancingScheme": "INTERNAL_MANAGED"
}
```

Create the URL map by making a `POST` request to the
[`urlMaps.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/urlMaps/insert.md.txt),
replacing `PROJECT_ID` with your project ID.

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/urlMaps

{
"name": "l7-ilb-map",
"defaultService": "projects/PROJECT_ID/global/backendServices/BACKEND_SERVICE_NAME"
}
```

**For HTTP**:

Create the target HTTP proxy by making a `POST` request to the
[`targetHttpProxies.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/targetHttpProxies/insert.md.txt),
replacing `PROJECT_ID` with your project ID.

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/targetHttpProxy

{
"name": "l7-ilb-proxy",
"urlMap": "projects/PROJECT_ID/global/urlMaps/l7-ilb-map"
}
```

Create the forwarding rule by making a `POST` request to the
[`forwardingRules.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/globalForwardingRules/insert.md.txt),
replacing `PROJECT_ID` with your project ID.

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/forwardingRules

{
"name": "FWRULE_A",
"IPAddress": "10.1.2.99",
"IPProtocol": "TCP",
"portRange": "80-80",
"target": "projects/PROJECT_ID/global/targetHttpProxies/l7-ilb-proxy",
"loadBalancingScheme": "INTERNAL_MANAGED",
"subnetwork": "projects/PROJECT_ID/regions/REGION_A/subnetworks/SUBNET_A",
"network": "projects/PROJECT_ID/global/networks/NETWORK",
"networkTier": "PREMIUM"
}
```

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/forwardingRules

{
"name": "gil7forwarding-rule-b",
"IPAddress": "10.1.3.99",
"IPProtocol": "TCP",
"portRange": "80-80",
"target": "projects/PROJECT_ID/global/targetHttpProxies/l7-ilb-proxy",
"loadBalancingScheme": "INTERNAL_MANAGED",
"subnetwork": "projects/PROJECT_ID/regions/REGION_B/subnetworks/SUBNET_B",
"network": "projects/PROJECT_ID/global/networks/NETWORK",
"networkTier": "PREMIUM"
}
```

**For HTTPS**:

Read the certificate and private key files, and then create the SSL
certificate. The following example shows
how to do this with Python.

```python
from pathlib import Path
from pprint import pprint
from typing import Union

from googleapiclient import discovery


def create_regional_certificate(
    project_id: str,
    region: str,
    certificate_file: Union[str, Path],
    private_key_file: Union[str, Path],
    certificate_name: str,
    description: str = "Certificate created from a code sample.",
) -> dict:
    """
    Create a regional SSL self-signed certificate within your Google Cloud project.

    Args:
        project_id: project ID or project number of the Cloud project you want to use.
        region: name of the region you want to use.
        certificate_file: path to the file with the certificate you want to create in your project.
        private_key_file: path to the private key you used to sign the certificate with.
        certificate_name: name for the certificate once it's created in your project.
        description: description of the certificate.

        Returns:
        Dictionary with information about the new regional SSL self-signed certificate.
    """
    service = discovery.build("compute", "v1")

    # Read the cert into memory
    with open(certificate_file) as f:
        _temp_cert = f.read()

    # Read the private_key into memory
    with open(private_key_file) as f:
        _temp_key = f.read()

    # Now that the certificate and private key are in memory, you can create the
    # certificate resource
    ssl_certificate_body = {
        "name": certificate_name,
        "description": description,
        "certificate": _temp_cert,
        "privateKey": _temp_key,
    }
    request = service.regionSslCertificates().insert(
        project=project_id, region=region, body=ssl_certificate_body
    )
    response = request.execute()
    pprint(response)

    return response

```

Create the target HTTPS proxy by making a `POST` request to the
[`targetHttpsProxies.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/targetHttpsProxies/insert.md.txt),
replacing `PROJECT_ID` with your project ID.

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/targetHttpsProxy

{
"name": "l7-ilb-proxy",
"urlMap": "projects/PROJECT_ID/global/urlMaps/l7-ilb-map",
"sslCertificates": /projects/PROJECT_ID/global/sslCertificates/SSL_CERT_NAME
}
```

Create the forwarding rule by making a `POST` request to the
[`globalForwardingRules.insert` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/globalForwardingRules/insert.md.txt),
replacing `PROJECT_ID` with your project ID.

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/forwardingRules

{
"name": "FWRULE_A",
"IPAddress": "10.1.2.99",
"IPProtocol": "TCP",
"portRange": "80-80",
"target": "projects/PROJECT_ID/global/targetHttpsProxies/l7-ilb-proxy",
"loadBalancingScheme": "INTERNAL_MANAGED",
"subnetwork": "projects/PROJECT_ID/regions/REGION_A/subnetworks/SUBNET_A",
"network": "projects/PROJECT_ID/global/networks/NETWORK",
"networkTier": "PREMIUM"
}
```

```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/forwardingRules

{
"name": "FWRULE_B",
"IPAddress": "10.1.3.99",
"IPProtocol": "TCP",
"portRange": "80-80",
"target": "projects/PROJECT_ID/global/targetHttpsProxies/l7-ilb-proxy",
"loadBalancingScheme": "INTERNAL_MANAGED",
"subnetwork": "projects/PROJECT_ID/regions/REGION_B/subnetworks/SUBNET_B",
"network": "projects/PROJECT_ID/global/networks/NETWORK",
"networkTier": "PREMIUM"
}
```

## Test the load balancer

### Create a VM instance to test connectivity

1. Create a client VM:

   ```
   gcloud compute instances create l7-ilb-client-a \
       --image-family=debian-12 \
       --image-project=debian-cloud \
       --network=NETWORK \
       --subnet=SUBNET_A \
       --zone=ZONE_A \
       --tags=allow-ssh
   ```

   ```
   gcloud compute instances create l7-ilb-client-b \
       --image-family=debian-12 \
       --image-project=debian-cloud \
       --network=NETWORK \
       --subnet=SUBNET_B \
       --zone=ZONE_B \
       --tags=allow-ssh
   ```
2. Use SSH to connect to each client instance.

   ```
   gcloud compute ssh l7-ilb-client-a --zone=ZONE_A
   ```

   ```
   gcloud compute ssh l7-ilb-client-b --zone=ZONE_B
   ```
3. Verify that the IP address is serving its hostname

   - Verify that the client VM can reach both IP addresses.
     The command returns the name of the backend VM which
     served the request:

     ```
     curl 10.1.2.99
     ```

     ```
     curl 10.1.3.99
     ```

     For HTTPS testing, replace `curl` with the following command line:

     ```
     curl -k 'https://DOMAIN_NAME:443' --connect-to DOMAIN_NAME:443:10.1.2.99:443
     ```

     ```
     curl -k 'https://DOMAIN_NAME:443' --connect-to DOMAIN_NAME:443:10.1.3.99:443
     ```

     Replace <var translate="no">DOMAIN_NAME</var> with your application domain name, for
     example, `test.example.com`.

     The `-k` flag causes curl to skip certificate validation.
   - Optional: Use the configured [DNS record](https://docs.cloud.google.com/load-balancing/docs/l7-internal/setting-up-l7-cross-reg-internal.md.txt#dns-policy-manager) to resolve
     the IP address closest to the client VM. For example, <var translate="no">DNS_NAME</var> can be
     `service.example.com`.

     ```
     curl DNS_NAME
     ```

#### Run 100 requests and confirm that they are load balanced

**For HTTP**:

<br />

```
  {
    RESULTS=
    for i in {1..100}
    do
        RESULTS="$RESULTS:$(curl --silent 10.1.2.99)"
    done
    echo ""
echo " Results of load-balancing to 10.1.2.99: "
    echo "***"
    echo "$RESULTS" | tr ':' '\n' | grep -Ev "^$" | sort | uniq -c
    echo
  }
  
```

<br />

<br />

```
  {
    RESULTS=
    for i in {1..100}
    do
      RESULTS="$RESULTS:$(curl --silent 10.1.3.99)"
    done
    echo ""
echo " Results of load-balancing to 10.1.3.99: "
    echo "***"
    echo "$RESULTS" | tr ':' '\n' | grep -Ev "^$" | sort | uniq -c
    echo
  }
  
```

<br />

**For HTTPS**:

In the following scripts, replace <var translate="no">DOMAIN_NAME</var> with your application
domain name, for example, `test.example.com`.

<br />

```
  {
    RESULTS=
    for i in {1..100}
    do
      RESULTS="$RESULTS:$(curl -k -s 'https://DOMAIN_NAME:443' --connect-to DOMAIN_NAME:443:10.1.2.99:443)"
    done
    echo ""
echo " Results of load-balancing to 10.1.2.99: "
    echo "***"
    echo "$RESULTS" | tr ':' '\n' | grep -Ev "^$" | sort | uniq -c
    echo
  }
  
```

<br />

<br />

```
  {
    RESULTS=
    for i in {1..100}
    do
       RESULTS="$RESULTS:$(curl -k -s 'https://DOMAIN_NAME:443' --connect-to DOMAIN_NAME:443:10.1.3.99:443)"
    done
    echo ""
echo " Results of load-balancing to 10.1.3.99: "
    echo "***"
    echo "$RESULTS" | tr ':' '\n' | grep -Ev "^$" | sort | uniq -c
    echo
  }
  
```

<br />

### Test failover

1. Verify failover to backends in the `REGION_A` region
   when backends in the `REGION_B` are unhealthy or
   unreachable. To simulate failover, remove all backends from
   `REGION_B`:

   ```
   gcloud compute backend-services remove-backend BACKEND_SERVICE_NAME \
      --balancing-mode=UTILIZATION \
      --instance-group=gl7-ilb-mig-b \
      --instance-group-zone=ZONE_B
   ```
2. Connect using SSH to a client VM in <var translate="no">REGION_B</var>.

   ```
   gcloud compute ssh l7-ilb-client-b \
      --zone=ZONE_B
   ```
3. Send requests to the load balanced IP address in the
   `REGION_B` region. The command output shows responses
   from backend VMs in `REGION_A`.

   In the following script, replace <var translate="no">DOMAIN_NAME</var> with your application
   domain name, for example, `test.example.com`.

   ```
   {
   RESULTS=
   for i in {1..100}
   do
     RESULTS="$RESULTS:$(curl -k -s 'https://DOMAIN_NAME:443' --connect-to DOMAIN_NAME:443:10.1.3.99:443)"
   done
   echo "***"
   echo "*** Results of load-balancing to 10.1.3.99: "
   echo "***"
   echo "$RESULTS" | tr ':' '\n' | grep -Ev "^$" | sort | uniq -c
   echo
   }
   ```

## Additional configuration options

This section expands on the configuration example to provide alternative and
additional configuration options. All of the tasks are optional. You can
perform them in any order.

### Enable session affinity

These procedures show you how to update a backend service for the example
regional internal Application Load Balancer or cross-region internal Application Load Balancer
so that the backend service
uses generated cookie affinity, header field affinity, or HTTP cookie affinity.

When generated cookie affinity is enabled, the load balancer issues a cookie
on the first request. For each subsequent request with the same cookie, the load
balancer directs the request to the same backend virtual machine (VM) instance
or endpoint. In this example, the cookie is named `GCILB`.

When header field affinity is enabled, the load balancer routes requests to
backend VMs or endpoints in a network endpoint group (NEG) based on the value of
the HTTP header named in the `--custom-request-header` flag.
Header field affinity is only valid if
the load balancing locality policy is either `RING_HASH` or `MAGLEV` and the
backend service's consistent hash specifies the name of the HTTP header.

When HTTP cookie affinity is enabled, the load balancer routes requests to
backend VMs or endpoints in a NEG, based on an HTTP cookie named in the
`HTTP_COOKIE` flag with the optional `--affinity-cookie-ttl` flag. If the client
doesn't provide the cookie in its HTTP request, the proxy generates
the cookie and returns it to the client in a `Set-Cookie` header. HTTP cookie
affinity is only valid if the load balancing locality policy is either
`RING_HASH` or `MAGLEV` and the backend service's consistent hash specifies the
HTTP cookie.

### Console

To enable or change session affinity for a backend service:

1. In the Google Cloud console, go to the **Load balancing** page.

   [Go to Load balancing](https://console.cloud.google.com/net-services/loadbalancing/loadBalancers/list)
2. Click **Backends**.
3. Click **gil7-backend-service** (the name of the backend service you created for this example), and then click **Edit**.
4. On the **Backend service details** page, click **Advanced
   configuration**.
5. Under **Session affinity**, select the type of session affinity you want.
6. Click **Update**.

### gcloud

Use the following Google Cloud CLI commands to update the
backend service to different types of session affinity:

```
    gcloud compute backend-services update gil7-backend-service \
        --session-affinity=[GENERATED_COOKIE | HEADER_FIELD | HTTP_COOKIE | CLIENT_IP] \
        --global
    
```

### API

To set session affinity, make a \`PATCH\` request to the
[`backendServices/patch` method](https://docs.cloud.google.com/compute/docs/reference/rest/v1/backendServices/patch.md.txt).

```
    PATCH https://compute.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/backendServices/gil7-backend-service
    {
    "sessionAffinity": ["GENERATED_COOKIE" | "HEADER_FIELD" | "HTTP_COOKIE" | "CLIENT_IP" ]
    }
    
```

### Restrict which clients can send traffic to the load balancer

You can restrict clients from connecting to an internal Application Load Balancer
forwarding rule VIP by configuring egress firewall rules on these clients. Set
these firewall rules on specific client VMs based on [service
accounts](https://docs.cloud.google.com/vpc/docs/firewalls#serviceaccounts) or
[tags](https://docs.cloud.google.com/vpc/docs/add-remove-network-tags.md.txt#adding_new_tags_to_vm_instances).

You can't use firewall rules to restrict inbound traffic to specific
internal Application Load Balancer forwarding rule VIPs. Any client on the same VPC
network and in the same region as the forwarding rule VIP can generally send
traffic to the forwarding rule VIP.

Additionally, all requests to backends come from proxies that use IP addresses in
the [proxy-only subnet](https://docs.cloud.google.com/load-balancing/docs/l7-internal.md.txt#proxy-only_subnet)
range. It isn't possible to create firewall rules that allow or deny ingress
traffic on these backends based on the forwarding rule VIP used by a client.

Here are some examples of how to use egress firewall rules to restrict traffic
to the load balancer's forwarding rule VIP.

### Console

To identify the client VMs, [tag the
specific VMs](https://docs.cloud.google.com/vpc/docs/add-remove-network-tags#adding_new_tags_to_vm_instances)
you want to restrict. These tags are used to associate firewall rules with
the tagged client VMs. Then, add the tag to the `TARGET_TAG`
field in the following steps.

Use either a single firewall rule or multiple rules to set this up.

#### Single egress firewall rule

You can configure one firewall egress rule to deny all egress
traffic going from tagged client VMs to a load balancer's VIP.

1. In the Google Cloud console, go to the **Firewall rules** page.

   [Go to Firewall rules](https://console.cloud.google.com/networking/firewalls/list)
2. Click **Create firewall rule** to create the rule to deny egress
   traffic from tagged client VMs to a load balancer's VIP.

   - **Name** : `fr-deny-access`
   - **Network** : `lb-network`
   - **Priority** : `100`
   - **Direction of traffic** : **Egress**
   - **Action on match** : **Deny**
   - **Targets** : **Specified target tags**
   - **Target tags** : `TARGET_TAG`
   - **Destination filter** : **IP ranges**
   - **Destination IP ranges** : `10.1.2.99`
   - **Protocols and ports** :
     - Choose **Specified protocols and ports**.
     - Select the **tcp** checkbox, and then enter `80` for the port number.
3. Click **Create**.

#### Multiple egress firewall rules

A more scalable approach involves setting two rules. A default, low-priority
rule that restricts all clients from accessing the load balancer's VIP. A
second, higher-priority rule that allows a subset of tagged clients to
access the load balancer's VIP. Only tagged VMs can access the
VIP.

1. In the Google Cloud console, go to the **Firewall rules** page.

   [Go to Firewall rules](https://console.cloud.google.com/networking/firewalls/list)
2. Click **Create firewall rule** to create the lower priority rule to deny
   access by default:

   - **Name** : `fr-deny-all-access-low-priority`
   - **Network** : `lb-network`
   - **Priority** : `200`
   - **Direction of traffic** : **Egress**
   - **Action on match** : **Deny**
   - **Targets** : **Specified target tags**
   - **Target tags** : `TARGET_TAG`
   - **Destination filter** : **IP ranges**
   - **Destination IP ranges** : `10.1.2.99`
   - **Protocols and ports** :
     - Choose **Specified protocols and ports**.
     - Select the **TCP** checkbox, and then enter `80` for the port number.
3. Click **Create**.

4. Click **Create firewall rule** to create the higher priority rule to
   allow traffic from certain tagged instances.

   - **Name** : `fr-allow-some-access-high-priority`
   - **Network** : `lb-network`
   - **Priority** : `100`
   - **Direction of traffic** : **Egress**
   - **Action on match** : **Allow**
   - **Targets** : **Specified target tags**
   - **Target tags** : `TARGET_TAG`
   - **Destination filter** : **IP ranges**
   - **Destination IP ranges** : `10.1.2.99`
   - **Protocols and ports** :
     - Choose **Specified protocols and ports**.
     - Select the **TCP** checkbox, and then enter `80` for the port number.
5. Click **Create**.

### gcloud

To identify the client VMs, [tag the
specific VMs](https://docs.cloud.google.com/vpc/docs/add-remove-network-tags#adding_new_tags_to_vm_instances)
you want to restrict. Then add the tag to the `TARGET_TAG`
field in these steps.

Use either a single firewall rule or multiple rules to set this up.

#### Single egress firewall rule

You can configure one firewall egress rule to deny all egress
traffic going from tagged client VMs to a load balancer's VIP.

```
gcloud compute firewall-rules create fr-deny-access \
    --network=lb-network \
    --action=deny \
    --direction=egress \
    --rules=tcp \
    --priority=100 \
    --destination-ranges=10.1.2.99 \
    --target-tags=TARGET_TAG
```

#### Multiple egress firewall rules

A more scalable approach involves setting two rules: a default, low-priority
rule that restricts all clients from accessing the load balancer's VIP, and a
second, higher-priority rule that allows a subset of tagged clients to access
the load balancer's VIP. Only tagged VMs can access the VIP.

1. Create the lower-priority rule:

   ```
   gcloud compute firewall-rules create fr-deny-all-access-low-priority \
       --network=lb-network \
       --action=deny \
       --direction=egress \
       --rules=tcp \
       --priority=200 \
       --destination-ranges=10.1.2.99
   ```
2. Create the higher priority rule:

   ```
   gcloud compute firewall-rules create fr-allow-some-access-high-priority \
       --network=lb-network \
       --action=allow \
       --direction=egress \
       --rules=tcp \
       --priority=100 \
       --destination-ranges=10.1.2.99 \
       --target-tags=TARGET_TAG
   ```

To use service accounts instead of tags to control access, use
the [`--target-service-accounts`
option](https://docs.cloud.google.com/sdk/gcloud/reference/compute/firewall-rules/create#--target-service-accounts)
instead of the `--target-tags` flag when creating firewall rules.

#### Scale restricted access to internal Application Load Balancer backends based on subnets

Maintaining separate firewall rules or adding new load-balanced IP addresses to
existing rules as described in the previous section becomes inconvenient as the
number of forwarding rules increases. One way to prevent this is to allocate
forwarding rule IP addresses from a reserved subnet.
Then, traffic from tagged instances or
service accounts can be allowed or blocked by using the reserved subnet as the
destination range for firewall rules. This lets you effectively control
access to a group of forwarding rule VIPs without having to maintain
per-VIP firewall egress rules.

Here are the high-level steps to set this up, assuming that you will create
all the other required load balancer resources separately.

### gcloud

1. Create a regional subnet to use to allocate load-balanced IP
   addresses for forwarding rules:

   ```
   gcloud compute networks subnets create l7-ilb-restricted-subnet \
       --network=lb-network \
       --region=us-west1 \
       --range=10.127.0.0/24
   ```
2. Create a forwarding rule that takes an address from the
   subnet. The following example uses the address `10.127.0.1` from the subnet
   created in the previous step.

   ```
   gcloud compute forwarding-rules create l7-ilb-forwarding-rule-restricted \
       --load-balancing-scheme=INTERNAL_MANAGED \
       --network=lb-network \
       --subnet=l7-ilb-restricted-subnet \
       --address=10.127.0.1 \
       --ports=80 \
       --global \
       --target-http-proxy=gil7-http-proxy
   ```
3. Create a firewall rule to restrict traffic destined for the range IP
   addresses in the forwarding rule subnet (`l7-ilb-restricted-subnet`):

   ```
   gcloud compute firewall-rules create restrict-traffic-to-subnet \
       --network=lb-network \
       --action=deny \
       --direction=egress \
       --rules=tcp:80 \
       --priority=100 \
       --destination-ranges=10.127.0.0/24 \
       --target-tags=TARGET_TAG
   ```

### Use the same IP address between multiple internal forwarding rules

For multiple internal forwarding rules to share the same internal IP address,
you must reserve the IP address and set its `--purpose` flag to
`SHARED_LOADBALANCER_VIP`.

### gcloud

```
gcloud compute addresses create SHARED_IP_ADDRESS_NAME \
    --region=REGION \
    --subnet=SUBNET_NAME \
    --purpose=SHARED_LOADBALANCER_VIP
```
If you need to redirect HTTP traffic to HTTPS, you can create two forwarding rules that use a common IP address. For more information, see [Set up
HTTP-to-HTTPS redirect for
internal Application Load Balancers](https://docs.cloud.google.com/load-balancing/docs/l7-internal/setting-up-http-to-https-redirect).

### Configure DNS routing policies

If your clients are in multiple regions, you might want to make your
cross-region internal Application Load Balancer accessible by using VIPs in these regions. You can use
DNS routing policies of type `GEO` to route client
traffic to the load balancer VIP in the region closest to the client. This
multi-region setup minimizes latency and network transit costs. In addition, it
lets you set up a DNS-based, global, load balancing solution that provides
resilience against regional outages.

Cloud DNS supports health checking and enables automatic failover when
the endpoints fail their health checks. During a failover, Cloud DNS
automatically adjusts the traffic split among the remaining healthy endpoints.
For more information, see [Manage DNS routing policies and health
checks](https://docs.cloud.google.com/dns/docs/zones/manage-routing-policies).

### gcloud

To create a DNS entry with a 30 second TTL, use the
[`gcloud dns record-sets create` command](https://docs.cloud.google.com/sdk/gcloud/reference/dns/record-sets/create).

```
gcloud dns record-sets create DNS_ENTRY --ttl="30" \
  --type="A" --zone="service-zone" \
  --routing-policy-type="GEO" \
  --routing-policy-data="REGION_A=gil7-forwarding-rule-a@global;REGION_B=gil7-forwarding-rule-b@global" \
  --enable-health-checking
```

Replace the following:

- `DNS_ENTRY`: DNS or domain name of the record-set

  For example, `service.example.com`
- `REGION_A` and `REGION_B`: the regions where you have configured the load balancer

### API

Create the DNS record by making a `POST` request to the
[`ResourceRecordSets.create` method](https://docs.cloud.google.com/dns/docs/reference/v1beta2/resourceRecordSets/create).
Replace <var translate="no">PROJECT_ID</var> with your project ID.

```
POST https://www.googleapis.com/dns/v1/projects/PROJECT_ID/managedZones/SERVICE_ZONE/rrsets
{
  "name": "DNS_ENTRY",
  "type": "A",
  "ttl": 30,
  "routingPolicy": {
    "geo": {
      "items": [
        {
          "location": "REGION_A",
          "healthCheckedTargets": {
            "internalLoadBalancers": [
              {
                "loadBalancerType": "globalL7ilb",
                "ipAddress": "IP_ADDRESS",
                "port": "80",
                "ipProtocol": "tcp",
                "networkUrl": "https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/networks/lb-network",
                "project": "PROJECT_ID"
              }
            ]
          }
        },
        {
          "location": "REGION_B",
          "healthCheckedTargets": {
            "internalLoadBalancers": [
              {
                "loadBalancerType": "globalL7ilb",
                "ipAddress": "IP_ADDRESS_B",
                "port": "80",
                "ipProtocol": "tcp",
                "networkUrl": "https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/networks/lb-network",
                "project": "PROJECT_ID"
              }
            ]
          }
        }
      ]
    }
  }
}
```

### Update client HTTP keepalive timeout

The load balancer created in the previous steps has been configured with a default value for the [client HTTP keepalive
timeout](https://docs.cloud.google.com/load-balancing/docs/l7-internal#http-keepalive-timeout).

To update the client HTTP keepalive timeout, use the following instructions.

### Console

1. In the Google Cloud console, go to the **Load balancing** page.

   [Go to Load balancing](https://console.cloud.google.com/networking/loadbalancing/add).
2. Click the name of the load balancer that you want to modify.
3. Click **Edit**.
4. Click **Frontend configuration**.
5. Expand **Advanced features** . For **HTTP keepalive timeout**, enter a timeout value.
6. Click **Update**.
7. To review your changes, click **Review and finalize** , and then click **Update**.

### gcloud

For an HTTP load balancer, update the target HTTP proxy by using the
[`gcloud compute target-http-proxies update` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/target-http-proxies/update):

```
      gcloud compute target-http-proxies update TARGET_HTTP_PROXY_NAME \
          --http-keep-alive-timeout-sec=HTTP_KEEP_ALIVE_TIMEOUT_SEC \
          --global
    
```

For an HTTPS load balancer, update the target HTTPS proxy by using the
[`gcloud compute target-https-proxies update` command](https://docs.cloud.google.com/sdk/gcloud/reference/compute/target-https-proxies/update):

```
      gcloud compute target-https-proxies update TARGET_HTTPS_PROXY_NAME \
          --http-keep-alive-timeout-sec=HTTP_KEEP_ALIVE_TIMEOUT_SEC \
          --global
    
```

Replace the following:

- `TARGET_HTTP_PROXY_NAME`: the name of the target HTTP proxy.
- `TARGET_HTTPS_PROXY_NAME`: the name of the target HTTPS proxy.
- `HTTP_KEEP_ALIVE_TIMEOUT_SEC`: the HTTP keepalive timeout value from 5 to 600 seconds.

### Enable outlier detection

You can enable [outlier
detection](https://docs.cloud.google.com/load-balancing/docs/negs/serverless-neg-concepts#outlier-detection)
on global backend services to identify unhealthy serverless NEGs and reduce the
number the requests sent to the unhealthy serverless NEGs.

Outlier detection is enabled on the backend service by using one of the following methods:

- The `consecutiveErrors` method (`outlierDetection.consecutiveErrors`), in which a `5*xx*` series HTTP status code qualifies as an error.
- The `consecutiveGatewayFailure` method (`outlierDetection.consecutiveGatewayFailure`), in which only the `502`, `503`, and `504` HTTP status codes qualify as an error.

Use the following steps to enable outlier detection for an existing backend
service. Note that even after enabling outlier detection, some requests can be
sent to the unhealthy service and return a `5*xx*` status code to
the clients. To further reduce the error rate, you can configure more aggressive
values for the outlier detection parameters. For more information, see the
[`outlierDetection` field](https://docs.cloud.google.com/compute/docs/reference/rest/v1/backendServices.md.txt#BackendService.FIELDS.outlier_detection).

### Console

1. In the Google Cloud console, go to the **Load balancing** page.

   [Go to Load balancing](https://console.cloud.google.com/networking/loadbalancing/)
2. Click the name of the load balancer whose backend service you want to
   edit.

3. On the **Load balancer details** page, click
   **Edit**.

4. On the **Edit cross-region internal Application Load Balancer** page, click **Backend configuration**.

5. On the **Backend configuration** page, click
   **Edit** for the backend service that
   you want to modify.

6. Scroll down and expand the **Advanced configurations** section.

7. In the **Outlier detection** section, select the **Enable** checkbox.

8. Click **Edit** to configure
   outlier detection.

   Verify that the following options are configured with these values:

   | Property | Value |
   |---|---|
   | **Consecutive errors** | 5 |
   | **Interval** | 1000 |
   | **Base ejection time** | 30000 |
   | **Max ejection percent** | 50 |
   | **Enforcing consecutive errors** | 100 |

   <br />


   In this example, the outlier detection analysis runs every one second. If
   the number of consecutive HTTP `5*xx*` status codes
   received by an
   Envoy proxy is five or more, the backend endpoint is ejected from the
   load-balancing pool of that Envoy proxy for 30 seconds. When the
   enforcing percentage is set to 100%, the backend service enforces the
   ejection of unhealthy endpoints from the load-balancing pools of those
   specific Envoy proxies every time the outlier detection analysis runs. If
   the ejection conditions are met, up to 50% of the backend endpoints from
   the load-balancing pool can be ejected.
9. Click **Save**.

10. To update the backend service, click **Update**.

11. To update the load balancer, on the **Edit cross-region internal Application Load Balancer** page, click **Update**.

### gcloud

1. Export the backend service into a YAML file.

   ```
   gcloud compute backend-services export BACKEND_SERVICE_NAME \
     --destination=BACKEND_SERVICE_NAME.yaml --global
   ```

   Replace `BACKEND_SERVICE_NAME` with the name of the
   backend service.
2. Edit the YAML configuration of the backend service to add the fields for
   outlier detection as highlighted in the following YAML configuration,
   in the `outlierDetection` section:


   In this example, the outlier detection analysis runs every one second. If
   the number of consecutive HTTP `5*xx*` status codes
   received by an
   Envoy proxy is five or more, the backend endpoint is ejected from the
   load-balancing pool of that Envoy proxy for 30 seconds. When the
   enforcing percentage is set to 100%, the backend service enforces the
   ejection of unhealthy endpoints from the load-balancing pools of those
   specific Envoy proxies every time the outlier detection analysis runs. If
   the ejection conditions are met, up to 50% of the backend endpoints from
   the load-balancing pool can be ejected.

   ```
   name: BACKEND_SERVICE_NAME
   backends:
   - balancingMode: UTILIZATION
     capacityScaler: 1.0
     group: https://www.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION_A/networkEndpointGroups/SERVERLESS_NEG_NAME
   - balancingMode: UTILIZATION
     capacityScaler: 1.0
     group: https://www.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION_B/networkEndpointGroups/SERVERLESS_NEG_NAME_2
   outlierDetection:
     baseEjectionTime:
       nanos: 0
       seconds: 30
     consecutiveErrors: 5
     enforcingConsecutiveErrors: 100
     interval:
       nanos: 0
       seconds: 1
     maxEjectionPercent: 50
   port: 80
   selfLink: https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendServices/BACKEND_SERVICE_NAME
   sessionAffinity: NONE
   timeoutSec: 30
   ...
   ```

   Replace the following:
   - `BACKEND_SERVICE_NAME`: the name of the backend service
   - `PROJECT_ID`: the ID of your project
   - `REGION_A` and `REGION_B`: the regions where the load balancer has been configured.
   - `SERVERLESS_NEG_NAME`: the name of the first serverless NEG
   - `SERVERLESS_NEG_NAME_2`: the name of the second serverless NEG
3. Update the backend service by importing the latest configuration.

   ```
   gcloud compute backend-services import BACKEND_SERVICE_NAME \
     --source=BACKEND_SERVICE_NAME.yaml --global
   ```

   Outlier detection is now enabled on the backend service.

## What's next

- [Convert Application Load Balancer to IPv6](https://docs.cloud.google.com/load-balancing/docs/https/convert-applb-dualstack.md.txt)
- [Internal Application Load Balancer overview](https://docs.cloud.google.com/load-balancing/docs/l7-internal.md.txt)
- [Proxy-only subnets for Envoy-based load balancers](https://docs.cloud.google.com/load-balancing/docs/proxy-only-subnets.md.txt)
- [Manage certificates](https://docs.cloud.google.com/certificate-manager/docs/certificates.md.txt)
- [Clean up a load balancing setup](https://docs.cloud.google.com/load-balancing/docs/cleaning-up-lb-setup.md.txt)