This feature lets you select a minimum CPU platform for your Managed Service for Apache Spark clusters. For example, when you create a cluster, you can select Intel Skylake CPUs for the cluster nodes to improve job performance.
Before using this feature, see Available regions & zones, which lists
the platforms available in each zone. You can also run the
gcloud compute zones describe zone command to list
the availableCpuPlatforms in the cluster zone. For more information, see
Specifying a minimum CPU platform for VM instances.
Use minimum CPU platform
Console
To specify a minimum CPU platform for cluster nodes:
- Open the Create cluster page.
- In the Worker configuration section, under CPU platform and GPU, confirm or select a CPU platform.
- By default, driver (master) node settings are the same as primary worker settings. Under Additional configuration, you can edit Driver node to clear the Default driver node to the same as primary worker checkbox, then specify driver node settings.
gcloud CLI command
To specify a minimum CPU platform for your cluster's master and/or worker (including
preemptible) nodes, pass one or both of the following flags to the
gcloud dataproc clusters create
command:
gcloud dataproc clusters create \ --region=region \ --master-min-cpu-platform=cpu-platform-name \ --worker-min-cpu-platform=cpu-platform-name \ other args ...
REST API
To specify a minimum CPU platform for your cluster's master, primary worker,
and/or secondary (preemptible) worker nodes, set the minCpuPlatform field in
InstanceGroupConfig
for your master, primary worker, and/or secondary (preemptible) instances in your
cluster.create
request.