Managed Service for Apache Spark shared responsibility model

Running business-critical workloads on Managed Service for Apache Spark requires multiple parties to carry different responsibilities. While not an exhaustive list, this page lists the responsibilities for Google and the customer.

Managed Service for Apache Spark: Google responsibilities

• Protecting the underlying infrastructure, including hardware, firmware, kernel, OS, storage, network, and more. This includes:

  • encrypting data at rest by default
  • providing additional customer-managed disk encryption
  • encrypting data in transit
  • using custom-designed hardware
  • laying private network cables
  • protecting data centers from physical access
  • protecting the bootloader and kernel against modification using Shielded Nodes
  • providing network protection with VPC Service Controls
  • following secure software development practices

• Releasing security patches for Managed Service for Apache Spark images . This includes:

  • patches for the base operating systems included in Managed Service for Apache Spark images (Ubuntu, Debian, and Rocky Linux)
  • patches and fixes available for the open source components included in Managed Service for Apache Spark images
    Security patches may only be available for operating system versions or open source software that are included in the most recent version of Managed Service for Apache Spark images. Leveraging the latest Managed Service for Apache Spark image version available is a customer responsibility.

• Providing Google Cloud integrations for Connect, Identity and Access Management, Cloud Audit Logs, Cloud Key Management Service, Security Command Center, and others.

• Restricting and logging Google administrative access to customer clusters for contractual support purposes with Access Transparency and Access Approval

• Recommending best practices for configuring Managed Service for Apache Spark and the open source components included in Managed Service for Apache Spark images

Managed Service for Apache Spark: Customer responsibilities

• Maintaining your workloads, including your application code, custom images, data, IAM policy, and clusters that you run

• Running clusters on up-to-date Managed Service for Apache Spark images by leveraging the latest subminor image version, promptly refreshing your custom images, and migrating to the most recent minor image version as soon as it is feasible. Image metadata includes a previous-subminor label, which is set to true if the cluster is not using the latest subminor image version. For information on how to view image metadata, see Important notes about versioning.

• Providing Google with environmental details when requested for troubleshooting purposes

• Following best practices for the configuration of Managed Service for Apache Spark and other Google Cloud services, and for the configuration of open source components included in Managed Service for Apache Spark images