This document describes a typical Google Cloud VMware Engine architecture in
Google Cloud. It also lists the security best practices that are
applicable to VMware Engine workloads and describes when you
would use specific Google Cloud services.

## Architecture

The following diagram shows the Google Cloud services in
a typical VMware Engine architecture.

![Sample architecture for VMware Engine.](https://docs.cloud.google.com/static/docs/security/images/sec-gcve-architecture.svg)

This diagram includes the following:

- [Backup and DR Service](https://docs.cloud.google.com/backup-disaster-recovery/docs/concepts/backup-dr) is a
  managed service that provides backup and recovery of workloads running in
  VMware Engine.

- [BigQuery](https://docs.cloud.google.com/bigquery/docs/introduction) provides data warehousing and
  analytic capabilities for data that's generated by the applications and
  databases that run on VMware Engine VMs.

- [Cloud Audit Logs](https://docs.cloud.google.com/logging/docs/audit) tracks the actions that your users take
  in your environment, which enhances your troubleshooting, auditing, and
  incident response capabilities.

- [Cloud Billing](https://docs.cloud.google.com/billing/docs/concepts) dashboards and alerts let you
  review usage and billing of VMware Engine workloads.

- [Cloud Identity](https://docs.cloud.google.com/identity/docs/overview) unifies identity, access,
  application, and management for Google Cloud.

- [Cloud Load Balancing](https://docs.cloud.google.com/load-balancing/docs/load-balancing-overview) can be
  used with Hybrid Network Endpoint Groups (NEGs) to distribute traffic to
  applications that run on VMware Engine VMs.

- [Cloud Storage](https://docs.cloud.google.com/storage/docs/introduction) stores data, including backup data,
  for VMware Engine VMs and workloads.

- [Compute Engine](https://docs.cloud.google.com/compute/docs/overview) can run applications that
  VMware Engine workloads interact with.

- [Cloud DNS](https://docs.cloud.google.com/dns/docs/overview) registers, manages, and serves your
  domain.

- [Google Cloud Armor](https://docs.cloud.google.com/armor/docs/cloud-armor-overview) provides DDoS protection
  and WAF capabilities for web applications hosted in
  VMware Engine and applications that are exposed using
  Cloud Load Balancing.

- [Google Kubernetes Engine](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/kubernetes-engine-overview)
  lets you run Kubernetes clusters on your VMware infrastructure within
  VMware Engine.

- [Identity and Access Management (IAM)](https://docs.cloud.google.com/iam/docs/overview) controls who can perform
  specific actions on VMware Engine and resources, such as creating,
  editing, or deleting them.

- [Organization Policy Service](https://docs.cloud.google.com/resource-manager/docs/organization-policy/overview)
  centrally manages and enforces policies across your Google Cloud
  environment. Organization Policy helps to ensure consistent configuration
  and security compliance across the projects and resources within your
  organization.

- [Resource Manager](https://docs.cloud.google.com/resource-manager/docs/cloud-platform-resource-hierarchy) helps
  you group and manage logical components of your VMware Engine
  workloads.

- [Secret Manager](https://docs.cloud.google.com/secret-manager/docs/overview) helps you protect
  the sensitive data and credentials that are used in
  VMware Engine projects.

- [Security Command Center](https://docs.cloud.google.com/security-command-center/docs/security-command-center-overview)
  helps you protect your cloud organization, your VMware workloads, and the data
  that you store on Google Cloud. Security Command Center provides the
  following:

  - Centralized security management
  - Threat detection and incident response
  - Automated security assessments
  - Compliance and regulatory reporting
  - Security recommendations and best practices
- [Virtual Private Cloud (VPC)](https://docs.cloud.google.com/vpc/docs/overview) isolates your resources from
  the internet in a secure environment. This network configuration helps protect
  sensitive data and workloads from unauthorized access and potential
  cyberattacks.

- [Cloud VPN](https://docs.cloud.google.com/network-connectivity/docs/vpn/concepts/overview) or
  [Cloud Interconnect](https://docs.cloud.google.com/network-connectivity/docs/interconnect/concepts/overview)
  lets you establish a secure network connection between your on-premises
  infrastructure and your VMware Engine environment.
  Cloud VPN or Cloud Interconnect helps enable seamless data
  transfer and communication between your private network and Google Cloud
  resources.

## Best practices for VMware Engine workloads

This section provides links to the best practices for workloads
that use VMware Engine.

- [Recommended user groups and IAM roles](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/recommended-iam-groups)
- Secure enterprise foundation best practices

  - Authentication and authorization best practices

    - [Disable automatic IAM grants for default service accounts](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#disable-automatic-iam-grants-for-default-service-accounts)
    - [Block the creation of external service account keys](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#block-the-creation-of-external-service-account-keys)
    - [Block service account key uploads](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#block-service-account-key-uploads)
    - [Configure separation of duties for organization policy administrators](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#configure-separation-of-duties-for-organization-policy-administrators)
    - [Enable two-step verification for super admin accounts](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#enable-two-step-verification-for-super-admin-accounts)
    - [Enforce two-step verification on the super admin organization unit](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#enforce-two-step-verification-on-the-super-admin-organization-unit)
    - [Create an exclusive email address for the primary super admin](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#create-an-exclusive-email-address-for-the-primary-super-admin)
    - [Create redundant administrator accounts](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#create-redundant-administrator-accounts)
    - [Implement tags to efficiently assign IAM policies and organization policies](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#implement-tags-to-efficiently-assign-iam-policies-and-organization-policies)
    - [Audit high-risk changes to IAM](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#audit-high-risk-changes-to-iam)
    - [Block access to Cloud Shell for Cloud Identity managed user accounts](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#block-access-to-cloud-shell-for-cloud-identity-managed-user-accounts)
    - [Configure Context-Aware Access for Google consoles](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#configure-context-aware-access-for-google-consoles)
    - [Block account self-recovery for super admin accounts](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#block-account-self-recovery-for-super-admin-accounts)
    - [Turn off unused Google services](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#turn-off-unused-google-services)
    - [Use Privileged Access Manager](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#use-privileged-access-manager)
  - Organization best practices

    - [Restrict TLS versions supported by Google APIs](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#restrict-tls-versions-supported-by-google-apis)
    - [Restrict authorized principals](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#restrict-authorized-principals)
    - [Restrict resource service usage](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#restrict-resource-service-usage)
    - [Restrict resource locations](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#restrict-resource-locations)
  - Networking best practices

    - [Block default network creation](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#block-default-network-creation)
    - [Enable DNS Security Extensions](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#enable-dns-security-extensions)
    - [Enable the service scope restriction in Access Context Manager access policies](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#enable-the-service-scope-restriction-in-access-context-manager-access-policies)
    - [Restrict APIs within VPC Service Controls service perimeters](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#restrict-apis-within-vpc-service-controls-service-perimeters)
    - [Use zonal DNS](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#use-zonal-dns)
    - [Enable Private Google Access](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#enable-private-google-access)
    - [Enable private service access for service producers](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#enable-private-service-access-for-service-producers)
  - Logging, monitoring, and alerting best practices

    - [Share audit logs from Cloud Identity](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#share-audit-logs-from-cloud-identity)
    - [Use audit logs](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#use-audit-logs)
    - [Enable auditing of administrator activity](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#enable-auditing-of-administrator-activity)
    - [Enable VPC Flow Logs](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#enable-vpc-flow-logs)
    - [Enable Firewall Rules Logging](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#enable-firewall-rules-logging)
    - [Enable Data Access audit logs](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#enable-data-access-audit-logs)
    - [Configure billing alerts](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#configure-billing-alerts)
    - [Enable Access Transparency logs](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#enable-access-transparency-logs)
  - Key and secret management best practices

    - [Encrypt data at rest in Google Cloud](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#encrypt-data-at-rest-in-google-cloud)
    - [Use NIST-approved algorithms for encryption and decryption](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#use-nist-approved-algorithms-for-encryption-and-decryption)
    - [Set the purpose for Cloud Key Management Service keys](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#set-the-purpose-for-cloud-kms-keys)
    - [Ensure that CMEK settings are appropriate for secure BigQuery data warehouses](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#ensure-that-cmek-settings-are-appropriate-for-secure-bigquery-data-warehouses)
    - [Rotate encryption key every 90 days](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#rotate-encryption-key-every-90-days)
    - [Set up automatic secret rotation](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#set-up-automatic-secret-rotation)
    - [Restrict customer-managed encryption keys location](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#restrict-customer-managed-encryption-keys-location-)
    - [Use CMEK for Google Cloud services](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#use-cmek-for-google-cloud-services)
    - [Replicate secrets automatically](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#replicate-secrets-automatically)
  - Security posture and analytics best practices

    - [Enable Security Command Center at the organization level](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#enable-security-command-center-at-the-organization-level)
    - [Configure alerts from Security Command Center](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/secure-enterprise-foundations#configure-alerts-from-security-command-center)
- Infrastructure best practices

  - Compute best practices

    - [Define VM instances that can enable IP forwarding](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#define-vm-instances-that-can-enable-ip-forwarding)
    - [Disable VM-nested virtualization](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#disable-vm-nested-virtualization)
    - [Restrict external IP addresses on VMs](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#restrict-external-ip-addresses-on-vms)
    - [Define permitted external IP addresses for VM instances](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#define-permitted-external-ip-addresses-for-vm-instances)
    - [Require VPC connector for Cloud Run functions](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#require-vpc-connector-for-cloud-run-functions)
    - [Turn off external IP addresses for Dataflow jobs](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#turn-off-external-ip-addresses-for-dataflow-jobs)
    - [Use network tags for firewall rules](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#use-network-tags-for-firewall-rules)
  - VMware Engine best practices

    - [Limit Admin role assignments for VMware Engine](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#limit-admin-role-assignments-for-vmware-engine)
    - [Use the VMware Engine Service Viewer role for least privilege](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#use-the-vmware-engine-service-viewer-role-for-least-privilege)
    - [Use RBAC and least privilege for vCenter Server Appliance roles](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#use-rbac-and-least-privilege-for-vcenter-server-appliance-roles)
    - [Use identity federation for VMware users](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#use-identity-federation-for-vmware-users)
    - [Grant roles to groups instead of individuals for vCenter Server Appliance](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#grant-roles-to-groups-instead-of-individuals-for-vcenter-server-appliance)
    - [Don't assign the Cloud-Owner-Role to user groups in vSphere](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#dont-assign-the-cloud-owner-role-to-user-groups-in-vsphere)
    - [Avoid using default vCenter and NSX-T service accounts](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#avoid-using-default-vcenter-and-nsx-t-service-accounts)
    - [Rotate passwords for default vCenter and NSX-T service accounts every 90 days](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#rotate-passwords-for-default-vcenter-and-nsx-t-service-accounts-every-90-days)
    - [Use the NSX Gateway Firewall to segment north-south traffic](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#use-the-nsx-gateway-firewall-to-segment-north-south-traffic)
    - [Use the NSX Distributed Firewall to segment east-west traffic](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#use-the-nsx-distributed-firewall-to-segment-east-west-traffic)
    - [Create separate subnets for workloads with different security requirements](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#create-separate-subnets-for-workloads-with-different-security-requirements)
    - [Create a log sink to store VMware Engine audit logs](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#create-a-log-sink-to-store-vmware-engine-audit-logs)
    - [Collect VMware-level platform logs](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#collect-vmware-level-platform-logs)
    - [Monitor applications using Logging and Monitoring](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#monitor-applications-using-logging-and-monitoring)
    - [Create private clouds in regions that match your data-residency requirements](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#create-private-clouds-in-regions-that-match-your-data-residency-requirements)
    - [Implement a backup and disaster recovery strategy](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#implement-a-backup-and-disaster-recovery-strategy)
    - [Implement application-level encryption for VMware workloads](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#implement-application-level-encryption-for-vmware-workloads)
    - [Enable data-in-transit encryption on VMware vSAN clusters](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#enable-data-in-transit-encryption-on-vmware-vsan-clusters)
    - [Configure vSAN data-at-rest encryption to use CMEK](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#configure-vsan-data-at-rest-encryption-to-use-cmek)
    - [Rotate the keys used for vSAN data-at-rest encryption](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/infrastructure#rotate-the-keys-used-for-vsan-data-at-rest-encryption)
- Data management best practices

  - Storage best practices

    - [Block public access to Cloud Storage buckets](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#block-public-access-to-cloud-storage-buckets)
    - [Use uniform bucket-level access](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#use-uniform-bucket-level-access)
    - [Protect HMAC keys for service accounts](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#protect-hmac-keys-for-service-accounts)
    - [Detect enumeration of Cloud Storage buckets by service accounts](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#detect-enumeration-of-cloud-storage-buckets-by-service-accounts)
    - [Ensure Cloud Storage bucket retention policy uses Bucket Lock](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#ensure-cloud-storage-bucket-retention-policy-uses-bucket-lock)
    - [Set lifecycle rules for the SetStorageClass action](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#set-lifecycle-rules-for-the-setstorageclass-action)
    - [Set permitted regions for storage classes](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#set-permitted-regions-for-storage-classes)
    - [Enable lifecycle management for Cloud Storage buckets](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#enable-lifecycle-management-for-cloud-storage-buckets)
    - [Enable lifecycle management rules for Cloud Storage buckets](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#enable-lifecycle-management-rules-for-cloud-storage-buckets)
    - [Review and evaluate temporary holds on active objects](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#review-and-evaluate-temporary-holds-on-active-objects)
    - [Enforce retention policies on Cloud Storage buckets](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#enforce-retention-policies-on-cloud-storage-buckets)
    - [Enforce classification tags for Cloud Storage buckets](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#enforce-classification-tags-for-cloud-storage-buckets)
    - [Enforce log buckets for Cloud Storage buckets](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#enforce-log-buckets-for-cloud-storage-buckets)
    - [Configure deletion rules for Cloud Storage buckets](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#configure-deletion-rules-for-cloud-storage-buckets)
    - [Ensure isLive condition is False for deletion rules](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#ensure-islive-condition-is-false-for-deletion-rules)
    - [Enforce versioning for Cloud Storage buckets](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#enforce-versioning-for-cloud-storage-buckets)
    - [Enforce owners for Cloud Storage buckets](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#enforce-owners-for-cloud-storage-buckets)
    - [Enable logging of key Cloud Storage activities](https://docs.cloud.google.com/docs/security/security-best-practices-catalog/data-management#enable-logging-of-key-cloud-storage-activities)

## What's next

- [Learn more about VMware Engine](https://docs.cloud.google.com/vmware-engine/docs/overview).

- [Migrate to a Google Cloud VMware Engine platform](https://docs.cloud.google.com/architecture/blueprints/vmware-engine-blueprint).