About migrating to Google Cloud with Hybrid Subnets

Hybrid Subnets helps you migrate workloads from another network—the source network—to a Virtual Private Cloud (VPC) subnet without needing to change any IP addresses. By combining a subnet in the source network with a VPC subnet, you create a single logical subnet that lets you migrate individual workloads and virtual machine (VM) instances over time. After all workloads and VMs have migrated, you can decommission the source subnet.

Hybrid Subnets also supports migrating VMs from Google Cloud to an on-premises network or between two VPC networks.

A three-stage diagram illustrates migrating workloads from on-premises to Google Cloud by using Hybrid Subnets.
Figure 1. During migration, a hybrid subnet provides connectivity between workloads in a source network and a VPC network (click to enlarge).

Specifications

Hybrid Subnets has the following specifications.

  • Properties:
    • A hybrid subnet is a single logical subnet that combines a segment of a source network with a subnet in a VPC network.
    • Internal connectivity is maintained between all VMs and workloads in a hybrid subnet.
    • The source network can be an on-premises network or another VPC network. The segment can be a whole subnet or part of one.
    • The two parts of a hybrid subnet must be connected by a network connectivity product such as Cloud VPN or Cloud Interconnect.
    • The primary IPv4 address range of the VPC subnet must match the range of the segment of the source network that is used by the hybrid subnet.
  • VPC network configuration:
    • You must enable hybrid subnet routing to configure a VPC subnet as part of a hybrid subnet. When hybrid subnet routing is enabled, custom routes can conflict (overlap) with the primary and secondary IPv4 address ranges of subnets.
    • You use Cloud Router custom advertised routes to selectively advertise the IP addresses of VMs as you migrate them to the VPC subnet. To support proxy ARP and longest prefix matching, these routes must be more specific (have a longer subnet mask) than the hybrid subnet's IP address range. You can use /32 routes for individual VMs.
  • Source network configuration:
    • You must configure proxy ARP in the source network.
    • You must configure the source network to advertise the IP address range of the hybrid subnet.
A Cloud Router and an on-premises router handle routing between the two parts of a hybrid subnet that uses a shared, overlapping IP address range.
Figure 2. This diagram provides an overview of the steps to create a hybrid subnet and how a Cloud Router and an on-premises router route packets between the two parts of a hybrid subnet (click to enlarge).

Hybrid subnet routing

A hybrid subnet combines a subnet in a source network with a VPC subnet to create a single logical subnet. Workloads in both parts of the hybrid subnet maintain internal connectivity; a workload can send traffic to a destination in either part of the hybrid subnet as if it were local, regardless of the destination's location.

VPC network routing

During the subnet routes matching step of the VPC routing model, if a packet's destination matches a local or peering subnet route, Google Cloud attempts to deliver the packet using the matching subnet route. In a regular subnet, if the destination isn't associated with a running VM or internal forwarding rule, the packet is dropped, and all other routes are disregarded.

However, if hybrid subnet routing is enabled for the subnet, the subnet route becomes a hybrid subnet route, and the routing behavior is different:

  • If the packet is associated with a running VM instance or internal forwarding rule in the VPC subnet, Google Cloud delivers the packet based on the local hybrid subnet route.
  • If the packet isn't associated with a running VM or internal forwarding rule in the VPC subnet, Google Cloud uses a special routing process for unmatched resources. This process includes checking for custom dynamic and static routes that overlap with the hybrid subnet's range. In a properly configured hybrid subnet, packets are routed to the source network by using a local dynamic route that the Cloud Router learns for the source subnet.
In a hybrid subnet, packet A is routed locally, and packet B is routed to an on-premises destination.
Figure 3. VMs in a hybrid subnet can reach destinations in an on-premises subnet and a VPC network subnet, even though both subnets share the same overlapping IP address range (click to enlarge).

For example, in figure 3, packet A is routed to a VM in the VPC part of a hybrid subnet by using a local hybrid subnet route. Packet B's destination isn't associated with a running VM or internal forwarding rule in the VPC part of the hybrid subnet, so Google Cloud checks for conflicting custom routes. A match is found, and Google Cloud uses the overlapping custom dynamic route to deliver Packet B to the source network.

Source network routing

When a workload in the source network sends a packet to a destination in the hybrid subnet's IP address range, the source network's router or first-hop device performs a routing table lookup.

If the destination is associated with a workload in the source network, the router does not intervene with proxy ARP. The destination receives the ARP request and responds with its own MAC address, and the packet is delivered locally.

If the destination is in the VPC part of the hybrid subnet, and the router has learned a dynamic route for the destination that is more specific than the local subnet route, the router selects the dynamic route by using longest prefix matching. This triggers the router's proxy ARP functionality. The router responds with its own MAC address and routes the packet to the Cloud Router in the VPC network.

An on-premises router uses proxy ARP to route a packet from a workload in a source network to a VM that has migrated to Google Cloud.
Figure 4. An on-premises router uses proxy ARP to route a packet from a workload in a source network to a VM that has migrated to Google Cloud (click to enlarge).

Limitations

Hybrid Subnets has the following limitations.

  • Resource limitations:

    These resource limitations are not enforced by Google Cloud limits or quotas. Exceeding them might cause connectivity and stability issues.

  • Unsupported traffic and routes:

    • Packets are dropped if the next hop of a conflicting route is in a different region than the hybrid subnet.
    • The following route types aren't supported as conflicting routes:
      • System-generated default routes
      • Policy-based routes
      • Static routes that have network tags
      • Routes with destinations that contain or are broader than the hybrid subnet route
    • Network Connectivity Center is not fully supported with Hybrid Subnets. You can configure a VPC network that contains a hybrid subnet to be a spoke of a Network Connectivity Center hub. However, traffic from connected spokes to the hybrid subnet's IP address range has unpredictable routing behavior.
    • Hybrid NAT is not supported with Hybrid Subnets. While you can configure a hybrid subnet to use hybrid NAT, the feature isn't applied to traffic that is affected by hybrid subnet routing.
    • Hybrid subnet routing doesn't apply to IPv6 traffic.
    • Broadcast and multicast traffic within a hybrid subnet are not supported.
    • You can't use Layer 3 Partner Interconnect connections that don't support announcing /32 routes with Hybrid Subnets.
    • A hybrid subnet's Cloud Router can't exceed the maximum number of custom advertised routes per BGP session.
    • Workloads in the source network can't reach Google APIs and services by using Private Google Access.
    • Workloads in the source network can't reach Private Service Connect endpoints for Google APIs.

  • Unsupported migration scenarios:

    • Hybrid Subnets doesn't support migrating workloads from other cloud service providers.
    • Hybrid Subnets doesn't support migrating VMs from an Azure or AWS source.
    • Hybrid Subnets doesn't support site-to-site data transfer.
    • Hybrid Subnets doesn't support Google Cloud VMware Engine as the migration target. If VMware Engine is your migration target, we recommend migrating VMware VMs using VMware HCX.

  • Other limitations:

    • Hybrid Subnets doesn't detect IP address conflicts between the source network and VPC parts of a hybrid subnet. Ensure that each IP address (except the default gateway) is used only once.
    • Hybrid Subnets can't host workloads at the reserved IP addresses in IPv4 subnets.
    • Workloads in the source network can't be endpoints for hybrid connectivity network endpoint groups that use centralized health checks.
    • Cloud DNS inbound forwarding doesn't respond to DNS requests from workloads in the source network.

Migration options

Google recommends using Migrate to Virtual Machines with Hybrid Subnets to automate the process of migrating VMs from a VMware source or from a Google Cloud VMware Engine source.

Alternatively, you can use third-party migration tools with Hybrid Subnets, as long as the requirements of Hybrid Subnets that are described in this document are fulfilled.

For information about planning a migration with Migrate to Virtual Machines, see Migration journey with Migrate to VMs.

For more information about migration options, see Migration resources.

For support with planning a migration to Google Cloud by using Hybrid Subnets, file a support case.

Considerations for using Hybrid Subnets

The following sections describe considerations for using Hybrid Subnets.

Proxy ARP and Hybrid Subnets

Hybrid Subnets requires proxy ARP to be configured on the source network's router or first-hop device (the point where a host first sends traffic that has a destination outside of its local network).

The first-hop device can be a router, virtual appliance, firewall, or a VM running a software solution such as choparp.

We recommend the following for using proxy ARP in your source network:

  • Consult the vendor of your source network fabric for best practices related to enabling proxy ARP and securing your network environment.
  • Disable proxy ARP after you have completed your migration to Google Cloud.

Regionality limitation

For a hybrid subnet to work correctly, conflicting routes (custom routes that overlap with the hybrid subnet's address range) must have all of their next hops in the same region as the hybrid subnet.

If a conflicting route has next hops in a different region:

  • If the route has a mix of local and remote next hops, traffic is dropped whenever ECMP selects a next hop in a remote region. This packet drop happens even if the packet also matches a less specific route that has next hops in the same region.
  • If the route has no next hops in the same region as the hybrid subnet, the packet is dropped.

Make sure that the following resources are located in the same region:

  • The VPC subnet that is configured as a hybrid subnet
  • The Cloud Router that learns routes to your source network
  • The HA VPN tunnels or VLAN attachments that provide hybrid connectivity

For example, suppose there is a hybrid subnet with the IP address range 192.0.2.0/24. The VPC subnet is in the region us-central1. The Cloud Router has learned two conflicting routes:

  • A custom route with destination range 192.0.2.0/25 and a next hop in the us-central1 region
  • A custom route with destination range 192.0.2.0/30 and a next hop in the us-west1 region.

A packet is sent to destination 192.0.2.2. This IP address isn't associated with a running VM or internal forwarding rule in the VPC subnet, so the route selection model selects the custom route that has the most specific destination, which is 192.0.2.0/30. This route doesn't have a next hop in the hybrid subnet's region, so the packet is dropped.

For more information, see unmatched resources in hybrid subnets.

VPC Network Peering

You can connect a hybrid subnet to a peer VPC network by using VPC Network Peering. The hybrid subnet's VPC network must be configured to export subnet and custom routes, and the peered VPC network must be configured to import them.

When the peered VPC network has programmed the routes, it can reach destinations within the hybrid subnet's IP address range, regardless of whether they exist in Google Cloud or the source network.

The routes won't be programmed for the peered network in the following cases:

  • A local subnet route in the peered network has a destination range that is identical to that of the imported route.
  • The dynamic routes per region per peering group quota is exceeded.
  • The two VPC networks are not directly peered. VPC Network Peering is not transitive.

If any of those conditions are true, the hybrid subnet won't work correctly from the perspective of the peered VPC network.

Network performance

Hybrid Subnets uses Layer 3 of the OSI model to route packets between the source network and VPC parts of a hybrid subnet. This approach helps Hybrid Subnets avoid challenges with latency, jitters, and throughput that can happen during migrations when some workloads exist on a source network but other workloads have migrated to the cloud.

In particular, avoiding Layer 2 tunneling helps prevent the performance degradation that's associated with the encapsulation and encryption of an additional Layer 2 overlay. Additionally, Layer 3 routing lets Hybrid Subnets avoid a common issue with Layer 2 tunneling, where traffic is sent to a central node before reaching destinations that can be close to the traffic's point of origin. This issue is sometimes called network tromboning.

Hybrid Subnets' approach to routing means that you can expect performance from a hybrid subnet that's similar to, or the same as, a network that doesn't use Hybrid Subnets.

Firewalls and Hybrid Subnets

Hybrid Subnets avoids challenges related to using firewalls with traffic that's encapsulated in Layer 2 overlays. For Layer 2 traffic, firewalls can only inspect packets at or beyond the overlay endpoints, unless you take specific measures such as transparent decryption or deep inspections of overlay traffic.

No special considerations are needed to use existing firewalls and firewall rules with Hybrid Subnets. However, you might need to Configure firewall rules to ensure that Google Cloud VMs can communicate with workloads in the source network.

Pricing

There is no additional charge for using Hybrid Subnets. However, you are charged for resources and network traffic in the VPC part of a hybrid subnet.

For more information, see Virtual Private Cloud pricing.

What's next