This document compares virtual machines (VMs) and bare metal instances to help you choose the right compute instance type for your workloads in Google Cloud.
When you deploy workloads on Compute Engine—either directly or by using managed services such as Google Kubernetes Engine (GKE) or Cluster Director—you can choose between two types of Compute Engine instances:
VMs: run on virtualized hardware managed by Google. They use a Kernel-based Virtual Machine (KVM) hypervisor, which lets multiple VMs securely share the resources of a single physical host server.
Bare metal instances: provide direct access to the underlying physical host server's CPU and memory resources, without a hypervisor in the middle. When you create a bare metal instance, you get exclusive access to the entire physical host server.
These two types of instances are collectively referred to as compute instances. You manage VMs and bare metal instances using the same pages in the Google Cloud console, and using the same commands in the Google Cloud CLI or Compute Engine API.
Use cases for VM and bare metal instances
Although VMs are the default and most flexible choice for most workloads, bare metal instances are designed for specific scenarios where the workloads require their own hypervisor layer or don't support virtualization.
Use cases for bare metal instances
Use a bare metal instance instead of a VM in the following scenarios:
- Third-party hypervisors: your workload needs to run its own virtualization stack (for example, VMware ESXi, KVM, or Xen) to manage isolated sandboxes or microVMs (using KVM or Firecracker), CI/CD pipelines, and private clouds (such as Nutanix AHV or VMware).
- Non-virtualizable workloads: your software has a licensing model that prohibits running on virtualized hardware or requires licensing per physical socket or core.
- High performance sensitivity: your workload can't tolerate the CPU, memory, or network latency overhead of a hypervisor, such as high-frequency trading platforms or real-time media processing.
- Direct hardware access: your workload requires direct access to hardware performance counters for low-level profiling, or to on-chip accelerators that aren't virtualized.
Use cases for VMs
For standard application servers, web servers, databases, and containerized workloads, use VMs as the most cost-effective option.
- Workloads with variable CPU usage: VMs let you scale resources up or down quickly.
- Rapid provisioning and operational speed: you can create, clone, or take snapshots of a VM in minutes or seconds, which is ideal for development, testing, and CI/CD pipelines.
- Minimal downtime: unless your VMs have accelerators attached, are Spot VMs, or use specific machine series, VMs support live migration. The live migration process helps minimize downtime by migrating a running VM from one host to another without rebooting the VM or interrupting the workload. Live migrations let Compute Engine perform scheduled hardware or software maintenance.
- Mixed or legacy applications: VMs can run standard multi-tenant, general-purpose enterprise software and web applications that don't require direct hardware manipulation.
- Lower entry cost: for small-scale, intermittent, or lower-traffic applications, sharing underlying physical hardware across multiple VMs costs less than paying for an entire dedicated server.
Hardware differences between VM and bare metal instances
The primary difference between VMs and bare metal instances lies in how they interact with the physical hardware. This difference affects performance, security, and the level of control you have over the host system.
CPU and memory access
The way CPU and memory are accessed differs significantly between VMs and bare metal instances, directly impacting performance and control:
- VMs: the KVM hypervisor abstracts the physical CPU and memory. The guest OS interacts with virtual CPUs (vCPUs) and virtualized memory. Although Compute Engine optimizes this abstraction to minimize overhead, there is a small virtualization cost. VMs support features such as nested virtualization on supported machine series.
- Bare metal instances: the OS runs directly on the physical CPU cores and accesses physical memory directly. This direct access eliminates hypervisor overhead and provides access to all CPU performance counters and built-in, on-CPU accelerators (such as Intel QAT, DLB, and IAA) that aren't fully exposed or virtualizable in a VM.
Third-generation and later machine series (such as C3, C4, H3, and H4D) use Titanium, Google's custom offload system. Titanium offload processors offload network processing, security policies, and disk I/O from the host CPU to provide consistent performance and low latency. Both VMs and bare metal instances use Titanium.
Network options
Both VMs and bare metal instances connect to your Virtual Private Cloud (VPC) networks, but they use different network interfaces and drivers:
- VMs: for first- and second-generation machine series, VirtIO is the default interface type. For third-generation and later machine series such as C3 and C4, Google Virtual NIC (gVNIC) is required. VMs support multiple network interfaces (up to 10 vNICs per VM).
- Bare metal instances: use the Infrastructure Data Path Function (IDPF) network interface. Bare metal instances don't support VirtIO or gVNIC. Most bare metal instances have a single network interface (except for multi-NIC machine series such as A5X, A4X Max, Z4D, and Z4M). However, you can use Dynamic Network Interfaces to configure additional VLAN sub-interfaces.
For workloads that require low network latency and high-throughput messaging (such as HPC), supported machine series can use Cloud RDMA (IRDMA) to bypass the OS kernel and copy data directly between memory and the network interface. Cloud RDMA is available for H4D VMs and for both VMs and bare metal instances in the Z4M machine series.
Instances created with machine types from the accelerator-optimized series (such as A3 Ultra, A4, A4X, A4X Max, and A5X) have attached GPUs. Compute Engine uses GPUDirect RDMA to provide high-bandwidth communication between the GPU and the MRDMA network interface.
Storage options
Both VMs and bare metal instances use block storage to store operating systems and application data.
Understand block storage
Block storage divides data into fixed-size chunks called blocks and stores those blocks on volumes that attach to your compute instances, such as virtual machines (VMs), containers, and bare metal instances. Your compute instances interact with block storage volumes like physical hard drives to read or write data. You can use block storage for boot disks that hold your OS and for data volumes that store your application files and databases. Across Google Cloud documentation and tools, the terms disk, volume, and block storage volume mean the same thing.
Google Cloud provides two categories of block storage: temporary block storage and durable block storage. You can attach both categories of block storage to a single compute instance. The two categories differ in data persistence, architecture, and performance:
Temporary block storage: resides on the physical server that hosts your compute instance. This local hardware connection delivers the highest input/output operations per second (IOPS) and the lowest latency among the block storage options on Google Cloud. However, temporary block storage is ephemeral: data persists across guest OS reboots, live migrations, and in-place maintenance restarts, but can be lost if you stop, suspend, or delete the instance, or if an unrecoverable host error occurs. For more information, see Local SSD data persistence.
This type of block storage is known as Local SSD or Titanium SSD disks. You can't use temporary block storage as a boot disk.
Durable block storage: connects to your compute instance over Google's network rather than through the physical host server. If you stop, suspend, restart, or delete your compute instance, or if the instance fails, then durable block storage preserves your data. Because durable volumes exist independently of your compute instances, you can attach non-boot data volumes to a running instance without stopping it, and you can detach a volume to keep your data or attach it to another instance even after you delete the original instance. You can replicate durable volumes across zones to help protect your data from zonal failures.
This type of block storage is known as Persistent Disk or Google Cloud Hyperdisk.
For information to help you choose the right type of block storage for your workload, see Choose a disk type.
Block storage for VMs and bare metal instances
When you use block storage with Compute Engine, note the following differences:
- VMs: support a wide range of storage options, including Persistent Disk volumes, Hyperdisk volumes, and Local SSD or Titanium SSD disks. First- and second-generation machine series use SCSI or NVMe, and third-generation and later machine series use only NVMe.
- Bare metal instances: use Hyperdisk volumes as their primary durable boot and data disks, and don't support Persistent Disk volumes. Some bare metal machine types also support Local SSD or Titanium SSD disks for high-performance, non-durable scratch space. Bare metal instances support only NVMe.
Comparison summary of hardware differences
The following table summarizes the hardware capability differences:
| Feature | VMs | Bare metal instances |
|---|---|---|
| Hypervisor | Yes (KVM) | No |
| CPU access | Virtualized (vCPUs) | Direct (Physical cores) |
| Network driver | VirtIO or gVNIC | IDPF |
| Max network interfaces | Up to 10 vNICs | Single network interface, except for A5X, A4X Max, Z4D, and Z4M. Dynamic NICs provide additional VLAN sub-interfaces. |
| Disk interface | SCSI or NVMe (support varies by machine series generation) | NVMe only |
| Block storage | Persistent Disk, Hyperdisk, Local SSD or Titanium SSD (on supported machine types) | Hyperdisk, Local SSD or Titanium SSD (on supported machine types) |
| Virtual Trusted Platform Module (vTPM) | Features such as Secure Boot, vTPM-enabled Measured Boot, and other Shielded VM features are available. | vTPM features aren't available for bare metal instances, except for A5X, A4X Max, and C4A, which support vTPM, but not Secure Boot or Measured Boot. |
OS management and maintenance differences
The choice between VMs and bare metal instances also affects how you manage the OS (or guest OS on VMs) and how Google maintains the underlying infrastructure.
Shared responsibility model
VMs offer a higher level of abstraction and simplify OS management. Bare metal instances give you direct, low-level control over the physical hardware and the OS:
- VMs: Google manages the host hardware, physical security, and the hypervisor. When you use Compute Engine directly, you manage the guest OS (including configuration, patching, and applications) and your data.
- Bare metal instances: Google manages the physical hardware and data center infrastructure. When you use Compute Engine directly, you manage the entire OS stack on the hardware, including hardware-level configurations and kernel modules.
If you deploy VMs or bare metal instances through a managed service such as GKE, Cluster Director, or Batch, then Google manages part or all of the OS configuration, patching, and node lifecycle for you. For more information, see Choose compute options.
Operating system images
OS image support for VMs and bare metal instances is similar, but bare metal instances don't support Windows images. For bare metal instances, you can use only Linux OS images that include the IDPF network driver. For details, see Supported OS images.
Transparent maintenance and host events
Google regularly maintains its infrastructure (hardware upgrades, patching). The impact on your instance depends on the instance type:
- VMs: many VMs support live migration. Google uses live migration technology to move running VMs to a new host server without downtime or rebooting. If live migration isn't supported (or if you choose not to use it), then Compute Engine stops and restarts the VM during maintenance.
- Bare metal instances: live migration isn't supported because the OS
runs directly on the hardware. You must set the host maintenance policy
for bare metal instances to
TERMINATE. During a maintenance event, Google stops the bare metal instance. If automatic restart is enabled, then Compute Engine restarts the instance after maintenance completes. Depending on the machine series, Compute Engine restarts the bare metal instance either:- On the same host (such as for X4 and X5 instances, or to preserve local Titanium SSD or Local SSD data)
- On another host in the same zone
For more information, see About host events and Transparent maintenance.
Performance monitoring
Monitoring the performance of bare metal instances differs from VMs because there is no hypervisor running on the host:
VMs: default CPU utilization metrics are available from the hypervisor. For more detailed guest OS telemetry, you can install the Ops Agent. By default, VMs don't expose low-level hardware performance counters (such as cache misses and branch mispredictions), unless you enable the performance monitoring unit (PMU) on supported machine series.
Bare metal instances: default hypervisor-based CPU utilization metrics aren't available. To monitor CPU utilization in Google Cloud Observability, install the Ops Agent and use the
OS Reported CPU %metric. Because bare metal instances run directly on physical hardware, you can also access all hardware performance counters and profile workload performance directly within the OS by using tools such asperf,sysstat, or vendor-specific profilers.
Physical placement and topology
Understanding how your VMs and bare metal instances are physically placed in Google's data centers helps when designing high-availability and low-latency systems.
Regions and zones
Google's infrastructure is organized in regions, geographical locations where a data center resides, and zones, isolated deployment areas within a region. VMs and bare metal instances are zonal resources; they are physically tied to the zone where you deploy them. For example, you can attach a boot disk when you create a compute instance only if the disk resides in the same zone as the compute instance.
For more information about how regions and zones affect compute resources in Google Cloud, see Regions and zones.
Data center hierarchy
Each compute instance runs on a physical server, a host, that exists in a server block. Each block belongs to a cluster in a zone. Each zone resides in one or more data centers.
Compute Engine uses the following terms when describing the physical location of each compute instance:
Cluster (
cluster): the global name of the cluster where your compute instance exists. A cluster is a high-level logical grouping of multiple hosts, which can span across several blocks, that work together as a single pool of resources.Block (
block): the organization-specific ID of the block where your compute instance exists. A block is a collection of multiple hosts grouped together.Sub-block (
subBlock): the organization-specific ID of the sub-block where your compute instance exists. A sub-block is a physical subdivision in a block, grouping hosts within a single physical enclosure.Host (
host): the ID of the physical server on which your compute instance runs. The host ID can be specific to your project or your organization, depending on the compute instance and its configuration.
By default, you manage the location of your compute instances only by specifying their zones. To control their physical placement within a zone:
Increase reliability: to place compute instances on separate, distinct hardware and reduce correlated failure risk, use spread placement policies.
Minimize network latency: to place compute instances closer to each other, use compact placement policies, workload policies for managed instance groups (MIGs), or the reservation-bound provisioning model.
Control physical host assignment: for creating VMs on dedicated physical servers and host-level affinity or anti-affinity, use sole-tenant nodes.
What's next
- Learn how to create and start a VM instance.
- Learn how to create and start a bare metal instance.
- Review the machine families resource and comparison guide to choose the machine series that meets your workload requirements.