This document compares virtual machines (VMs) and bare metal instances to
help you choose the right compute instance type for your workloads in
Google Cloud.

When you deploy workloads on Compute Engine---either directly or by
using managed services such as Google Kubernetes Engine (GKE) or
Cluster Director---you can choose between two types of
Compute Engine instances:

- **VMs**: run on virtualized hardware managed by Google. They use a
  Kernel-based Virtual Machine (KVM) hypervisor, which lets multiple VMs
  securely share the resources of a single physical host server.

- **Bare metal instances**: provide direct access to the underlying
  physical host server's CPU and memory resources, without a hypervisor in
  the middle. When you create a bare metal instance, you get exclusive
  access to the entire physical host server.

These two types of instances are collectively referred to as *compute
instances*. You manage VMs and bare metal instances using the same pages in the
Google Cloud console, and using the same commands in the
Google Cloud CLI or Compute Engine API.

## Use cases for VM and bare metal instances

Although VMs are the default and most flexible choice for most workloads,
bare metal instances are designed for specific scenarios where the workloads
require their own hypervisor layer or don't support virtualization.

### Use cases for bare metal instances

Use a bare metal instance instead of a VM in the following scenarios:

- **Third-party hypervisors**: your workload needs to run its own virtualization stack (for example, VMware ESXi, KVM, or Xen) to manage isolated sandboxes or microVMs (using KVM or Firecracker), CI/CD pipelines, and private clouds (such as Nutanix AHV or VMware).
- **Non-virtualizable workloads**: your software has a licensing model that prohibits running on virtualized hardware or requires licensing per physical socket or core.
- **High performance sensitivity**: your workload can't tolerate the CPU, memory, or network latency overhead of a hypervisor, such as high-frequency trading platforms or real-time media processing.
- **Direct hardware access**: your workload requires direct access to hardware performance counters for low-level profiling, or to on-chip accelerators that aren't virtualized.

### Use cases for VMs

For standard application servers, web servers, databases, and containerized
workloads, use VMs as the most cost-effective option.

- Workloads with variable CPU usage: VMs let you scale resources up or down quickly.
- Rapid provisioning and operational speed: you can create, clone, or take snapshots of a VM in minutes or seconds, which is ideal for development, testing, and CI/CD pipelines.
- Minimal downtime: unless your VMs have accelerators attached, are Spot VMs, or use [specific machine series](https://docs.cloud.google.com/compute/docs/instances/live-migration-process.md.txt#limitations), VMs support live migration. The live migration process helps minimize downtime by migrating a running VM from one host to another without rebooting the VM or interrupting the workload. Live migrations let Compute Engine perform scheduled hardware or software maintenance.
- Mixed or legacy applications: VMs can run standard multi-tenant, general-purpose enterprise software and web applications that don't require direct hardware manipulation.
- Lower entry cost: for small-scale, intermittent, or lower-traffic applications, sharing underlying physical hardware across multiple VMs costs less than paying for an entire dedicated server.

## Hardware differences between VM and bare metal instances

The primary difference between VMs and bare metal instances lies in how
they interact with the physical hardware. This difference affects performance,
security, and the level of control you have over the host system.

### CPU and memory access

The way CPU and memory are accessed differs significantly between VMs
and bare metal instances, directly impacting performance and control:

- **VMs** : the KVM hypervisor abstracts the physical CPU and memory. The guest OS interacts with virtual CPUs (vCPUs) and virtualized memory. Although Compute Engine optimizes this abstraction to minimize overhead, there is a small virtualization cost. VMs support features such as [nested virtualization](https://docs.cloud.google.com/compute/docs/instances/nested-virtualization/overview.md.txt) on supported machine series.
- **Bare metal instances** : the OS runs directly on the physical CPU cores and accesses physical memory directly. This direct access eliminates hypervisor overhead and provides access to all CPU performance counters and built-in, [on-CPU accelerators](https://docs.cloud.google.com/compute/docs/cpu-platforms.md.txt#accelerators) (such as Intel QAT, DLB, and IAA) that aren't fully exposed or virtualizable in a VM.

Third-generation and later machine series (such as C3, C4, H3, and H4D) use
Titanium, Google's custom offload system. Titanium
offload processors offload network processing, security policies, and disk
I/O from the host CPU to provide consistent performance and low latency.
Both VMs and bare metal instances use Titanium.

### Network options

Both VMs and bare metal instances connect to your
Virtual Private Cloud (VPC) networks, but they use different network interfaces
and drivers:

- **VMs** : for [first- and second-generation machine series](https://docs.cloud.google.com/compute/docs/machine-resource.md.txt#vm_terminology), VirtIO is the default interface type. For third-generation and later machine series such as C3 and C4, [Google Virtual NIC (gVNIC)](https://docs.cloud.google.com/compute/docs/networking/using-gvnic.md.txt) is required. VMs support multiple network interfaces (up to 10 vNICs per VM).
- **Bare metal instances** : use the [Infrastructure Data Path Function (IDPF)](https://docs.cloud.google.com/compute/docs/networking/using-idpf.md.txt) network interface. Bare metal instances don't support VirtIO or gVNIC. Most bare metal instances have a single network interface (except for multi-NIC machine series such as A5X, A4X Max, Z4D, and Z4M). However, you can use [Dynamic Network Interfaces](https://docs.cloud.google.com/vpc/docs/multiple-interfaces-concepts.md.txt#dynamic-nics) to configure additional VLAN sub-interfaces.

For workloads that require low network latency and high-throughput messaging
(such as HPC), supported machine series can use
[Cloud RDMA (IRDMA)](https://docs.cloud.google.com/compute/docs/networking/using-irdma.md.txt) to bypass
the OS kernel and copy data directly between memory and the network
interface. Cloud RDMA is available for H4D VMs and for both VMs and
bare metal instances in the Z4M machine series.

Instances created with machine types from the accelerator-optimized series
(such as A3 Ultra, A4, A4X, A4X Max, and A5X) have attached GPUs.
Compute Engine uses GPUDirect RDMA to provide
high-bandwidth communication between the GPU and the MRDMA network interface.

### Storage options

Both VMs and bare metal instances use block storage to store operating systems
and application data.

#### Understand block storage

Block storage divides data into fixed-size chunks called *blocks* and stores
those blocks on volumes that attach to your compute instances, such as virtual
machines (VMs), containers, and bare metal instances. Your compute instances
interact with block storage volumes like physical hard drives to read or write
data. You can use block storage for boot disks that hold your OS and for data
volumes that store your application files and databases. Across
Google Cloud documentation and tools, the terms *disk* , *volume* ,
and *block storage volume* mean the same thing.

Google Cloud provides two categories of block storage: temporary block
storage and durable block storage. You can attach both categories of block
storage to a single compute instance. The two categories differ in data
persistence, architecture, and performance:

- **Temporary block storage** : resides on the physical server that hosts your
  compute instance. This local hardware connection delivers the highest
  input/output operations per second (IOPS) and the lowest latency among the
  block storage options on Google Cloud. However, temporary block
  storage is ephemeral: data persists across guest OS reboots, live
  migrations, and in-place maintenance restarts, but can be lost if you
  stop, suspend, or delete the instance, or if an unrecoverable host error
  occurs. For more information, see
  [Local SSD data persistence](https://docs.cloud.google.com/compute/docs/disks/local-ssd.md.txt#data_persistence).

  This type of block storage is known as Local SSD or Titanium SSD
  disks. You can't use temporary block storage as a boot disk.
- **Durable block storage**: connects to your compute instance over Google's
  network rather than through the physical host server. If you stop, suspend,
  restart, or delete your compute instance, or if the instance fails, then
  durable block storage preserves your data. Because durable volumes exist
  independently of your compute instances, you can attach non-boot data
  volumes to a running instance without stopping it, and you can detach a
  volume to keep your data or attach it to another instance even after you
  delete the original instance. You can replicate durable volumes across
  zones to help protect your data from zonal failures.

  This type of block storage is known as Persistent Disk or Google Cloud Hyperdisk.

For information to help you choose the right type of block storage for
your workload, see [Choose a disk type](https://docs.cloud.google.com/compute/docs/disks.md.txt).

#### Block storage for VMs and bare metal instances

When you use block storage with Compute Engine, note the following
differences:

- **VMs**: support a wide range of storage options, including Persistent Disk volumes, Hyperdisk volumes, and Local SSD or Titanium SSD disks. First- and second-generation machine series use SCSI or NVMe, and third-generation and later machine series use only NVMe.
- **Bare metal instances**: use Hyperdisk volumes as their primary durable boot and data disks, and don't support Persistent Disk volumes. Some bare metal machine types also support Local SSD or Titanium SSD disks for high-performance, non-durable scratch space. Bare metal instances support only NVMe.

### Comparison summary of hardware differences

The following table summarizes the hardware capability differences:

| Feature | VMs | Bare metal instances |
| Hypervisor | Yes (KVM) | No |
| CPU access | Virtualized (vCPUs) | Direct (Physical cores) |
| Network driver | VirtIO or gVNIC | IDPF |
| Max network interfaces | Up to 10 vNICs | Single network interface, except for A5X, A4X Max, Z4D, and Z4M. Dynamic NICs provide additional VLAN sub-interfaces. |
| Disk interface | SCSI or NVMe (support varies by machine series generation) | NVMe only |
| Block storage | Persistent Disk, Hyperdisk, Local SSD or Titanium SSD (on supported machine types) | Hyperdisk, Local SSD or Titanium SSD (on supported machine types) |
| Virtual Trusted Platform Module (vTPM) | Features such as Secure Boot, vTPM-enabled Measured Boot, and other [Shielded VM](https://docs.cloud.google.com/compute/shielded-vm/docs/shielded-vm.md.txt) features are available. | vTPM features aren't available for bare metal instances, except for A5X, A4X Max, and C4A, which support vTPM, but not Secure Boot or Measured Boot. |
|---|---|---|

## OS management and maintenance differences

The choice between VMs and bare metal instances also affects how you
manage the OS (or guest OS on VMs) and how Google maintains the
underlying infrastructure.

### Shared responsibility model

VMs offer a higher level of abstraction and simplify OS management. Bare metal
instances give you direct, low-level control over the physical hardware and
the OS:

- **VMs**: Google manages the host hardware, physical security, and the hypervisor. When you use Compute Engine directly, you manage the guest OS (including configuration, patching, and applications) and your data.
- **Bare metal instances**: Google manages the physical hardware and data center infrastructure. When you use Compute Engine directly, you manage the entire OS stack on the hardware, including hardware-level configurations and kernel modules.

If you deploy VMs or bare metal instances through a managed service such as
GKE, Cluster Director, or Batch, then Google
manages part or all of the OS configuration, patching, and node lifecycle for
you. For more information, see
[Choose compute options](https://docs.cloud.google.com/docs/compute-area/choose-compute-options.md.txt).

### Operating system images

OS image support for VMs and bare metal instances is similar, but bare
metal instances don't support Windows images. For bare metal instances, you
can use only Linux OS images that include the
IDPF network driver. For details, see
[Supported OS images](https://docs.cloud.google.com/compute/docs/images/os-details.md.txt#networking).

### Transparent maintenance and host events

Google regularly maintains its infrastructure (hardware upgrades, patching).
The impact on your instance depends on the instance type:

- **VMs** : many VMs support live migration. Google uses live migration technology to move running VMs to a new host server without downtime or rebooting. If [live migration isn't supported](https://docs.cloud.google.com/compute/docs/instances/live-migration-process.md.txt#limitations) (or if you choose not to use it), then Compute Engine stops and restarts the VM during maintenance.
- **Bare metal instances** : live migration isn't supported because the OS runs directly on the hardware. You must set the host maintenance policy for bare metal instances to `TERMINATE`. During a maintenance event, Google stops the bare metal instance. If automatic restart is enabled, then Compute Engine restarts the instance after maintenance completes. Depending on the machine series, Compute Engine restarts the bare metal instance either:
  - On the same host (such as for X4 and X5 instances, or to preserve local Titanium SSD or Local SSD data)
  - On another host in the same zone

For more information, see
[About host events](https://docs.cloud.google.com/compute/docs/instances/host-maintenance-overview.md.txt) and
[Transparent maintenance](https://docs.cloud.google.com/compute/docs/regions-zones.md.txt#maintenance).

## Performance monitoring

Monitoring the performance of bare metal instances differs from VMs because
there is no hypervisor running on the host:

- **VMs** : default CPU utilization metrics are available from the
  hypervisor. For more detailed guest OS telemetry, you can install the
  [Ops Agent](https://docs.cloud.google.com/stackdriver/docs/solutions/agents/ops-agent.md.txt). By default,
  VMs don't expose low-level hardware performance counters (such as cache
  misses and branch mispredictions), unless you
  [enable the performance monitoring unit (PMU)](https://docs.cloud.google.com/compute/docs/pmu-overview.md.txt)
  on supported machine series.

- **Bare metal instances** : default hypervisor-based CPU utilization
  metrics aren't available. To monitor CPU utilization in Google Cloud Observability,
  install the Ops Agent and use the `OS Reported CPU %` metric. Because
  bare metal instances run directly on physical hardware, you can also
  access all hardware performance counters and profile workload
  performance directly within the OS by using tools such as `perf`,
  `sysstat`, or vendor-specific profilers.

## Physical placement and topology

Understanding how your VMs and bare metal instances are physically placed in
Google's data centers helps when designing high-availability and low-latency
systems.

### Regions and zones

Google's infrastructure is organized in *regions* , geographical locations where
a data center resides, and *zones*, isolated deployment areas within a region.
VMs and bare metal instances are zonal resources; they are physically
tied to the zone where you deploy them. For example, you can attach a boot
disk when you create a compute instance only if the disk resides in the same
zone as the compute instance.

For more information about how regions and zones affect compute resources in
Google Cloud, see [Regions and zones](https://docs.cloud.google.com/compute/docs/regions-zones.md.txt).

### Data center hierarchy

Each compute instance runs on a physical server, a *host*, that exists in a
server block. Each block belongs to a cluster in a zone. Each zone resides in
one or more data centers.

Compute Engine uses the following terms when describing the physical
location of each compute instance:

- **Cluster (`cluster`)** : the global name of the cluster where your compute
  instance exists. A *cluster* is a high-level logical grouping of multiple
  hosts, which can span across several blocks, that work together as a single
  pool of resources.

- **Block (`block`)** : the organization-specific ID of the block where your
  compute instance exists. A *block* is a collection of multiple hosts grouped
  together.

- **Sub-block (`subBlock`)** : the organization-specific ID of the sub-block
  where your compute instance exists. A *sub-block* is a physical subdivision
  in a block, grouping hosts within a single physical enclosure.

- **Host (`host`)**: the ID of the physical server on which your compute
  instance runs. The host ID can be specific to your project or your
  organization, depending on the compute instance and its configuration.

By default, you manage the location of your compute instances only by specifying
their zones. To control their physical placement within a zone:

- **Increase reliability** : to place compute instances on separate, distinct
  hardware and reduce correlated failure risk, use
  [spread placement policies](https://docs.cloud.google.com/compute/docs/instances/placement-policies-overview.md.txt#about-spread-policies).

- **Minimize network latency** : to place compute instances closer to
  each other, use [compact placement policies](https://docs.cloud.google.com/compute/docs/instances/placement-policies-overview.md.txt#about-compact-policies),
  [workload policies for managed instance groups (MIGs)](https://docs.cloud.google.com/compute/docs/instance-groups/about-workload-policies.md.txt),
  or the
  [reservation-bound provisioning model](https://docs.cloud.google.com/compute/docs/instances/about-reservation-bound-instances.md.txt).

- **Control physical host assignment** : for creating VMs on dedicated physical
  servers and host-level affinity or anti-affinity, use
  [sole-tenant nodes](https://docs.cloud.google.com/compute/docs/nodes/sole-tenant-nodes.md.txt).

## What's next

- Learn how to [create and start a VM instance](https://docs.cloud.google.com/compute/docs/instances/create-start-instance.md.txt).
- Learn how to [create and start a bare metal instance](https://docs.cloud.google.com/compute/docs/instances/create-vm-from-public-image.md.txt#baremetal).
- Review the [machine families resource and comparison guide](https://docs.cloud.google.com/compute/docs/machine-resource.md.txt) to choose the machine series that meets your workload requirements.