- Resource: Cluster
- NetworkResource
- NetworkReference
- NetworkResourceConfig
- NewNetworkConfig
- ExistingNetworkConfig
- StorageResource
- FilestoreReference
- BucketReference
- LustreReference
- StorageResourceConfig
- NewFilestoreConfig
- FileShareConfig
- Tier
- Protocol
- ExistingFilestoreConfig
- NewBucketConfig
- GcsAutoclassConfig
- TerminalStorageClass
- StorageClass
- GcsHierarchicalNamespaceConfig
- ExistingBucketConfig
- NewLustreConfig
- ExistingLustreConfig
- ComputeResource
- ComputeResourceConfig
- NewOnDemandInstancesConfig
- NewSpotInstancesConfig
- NewReservedInstancesConfig
- NewFlexStartInstancesConfig
- Orchestrator
- SlurmOrchestrator
- SlurmLoginNodes
- StorageConfig
- ComputeInstance
- BootDisk
- SlurmNodeSet
- ComputeInstanceSlurmNodeSet
- SlurmPartition
- Methods
Resource: Cluster
A collection of virtual machines and connected resources forming a high-performance computing cluster capable of running large-scale, tightly coupled workloads. A cluster combines a set a compute resources that perform computations, storage resources that contain inputs and store outputs, an orchestrator that is responsible for assigning jobs to compute resources, and network resources that connect everything together.
| JSON representation |
|---|
{ "name": string, "description": string, "labels": { string: string, ... }, "createTime": string, "updateTime": string, "reconciling": boolean, "networkResources": { string: { object ( |
| Fields | |
|---|---|
name |
Identifier. Relative resource name of the cluster, in the format |
description |
Optional. User-provided description of the cluster. |
labels |
Optional. Labels applied to the cluster. Labels can be used to organize clusters and to filter them in queries. An object containing a list of |
createTime |
Output only. Time that the cluster was originally created. Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: |
updateTime |
Output only. Time that the cluster was most recently updated. Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: |
reconciling |
Output only. Indicates whether changes to the cluster are currently in flight. If this is |
networkResources |
Optional. Network resources available to the cluster. Must contain at most one value. Keys specify the ID of the network resource by which it can be referenced elsewhere, and must conform to RFC-1034 (lower-case, alphanumeric, and at most 63 characters). An object containing a list of |
storageResources |
Optional. Storage resources available to the cluster. Keys specify the ID of the storage resource by which it can be referenced elsewhere, and must conform to RFC-1034 (lower-case, alphanumeric, and at most 63 characters). An object containing a list of |
computeResources |
Optional. Compute resources available to the cluster. Keys specify the ID of the compute resource by which it can be referenced elsewhere, and must conform to RFC-1034 (lower-case, alphanumeric, and at most 63 characters). An object containing a list of |
orchestrator |
Optional. Orchestrator that is responsible for scheduling and running jobs on the cluster. |
NetworkResource
A resource representing a network that connects the various components of a cluster together.
| JSON representation |
|---|
{ "config": { object ( |
| Fields | |
|---|---|
config |
Immutable. Configuration for this network resource, which describes how it should be created or imported. This field only controls how the network resource is initially created or imported. Subsequent changes to the network resource should be made via the resource's API and will not be reflected in the configuration. |
Union field reference. Reference to the network resource in Google Cloud. Exactly one of these fields will be populated based on the configured type of network resource. reference can be only one of the following: |
|
network |
Reference to a network in Google Compute Engine. |
NetworkReference
A reference to a VPC network in Google Compute Engine.
| JSON representation |
|---|
{ "network": string, "subnetwork": string } |
| Fields | |
|---|---|
network |
Output only. Name of the network, in the format |
subnetwork |
Output only. Name of the particular subnetwork being used by the cluster, in the format |
NetworkResourceConfig
Describes how a network resource should be initialized. Each network resource can either be imported from an existing Google Cloud resource or initialized when the cluster is created.
| JSON representation |
|---|
{ // Union field |
| Fields | |
|---|---|
Union field config. Particular type of configuration for this network resource. config can be only one of the following: |
|
newNetwork |
Optional. Immutable. If set, indicates that a new network should be created. |
existingNetwork |
Optional. Immutable. If set, indicates that an existing network should be imported. |
NewNetworkConfig
When set in a NetworkResourceConfig, indicates that a new network should be created.
| JSON representation |
|---|
{ "network": string, "description": string } |
| Fields | |
|---|---|
network |
Required. Immutable. Name of the network to create, in the format |
description |
Optional. Immutable. Description of the network. Maximum of 2048 characters. |
ExistingNetworkConfig
When set in a NetworkResourceConfig, indicates that an existing network should be imported.
| JSON representation |
|---|
{ "network": string, "subnetwork": string } |
| Fields | |
|---|---|
network |
Required. Immutable. Name of the network to import, in the format |
subnetwork |
Required. Immutable. Particular subnetwork to use, in the format |
StorageResource
A resource representing a form of persistent storage that is accessible to compute resources in the cluster.
| JSON representation |
|---|
{ "config": { object ( |
| Fields | |
|---|---|
config |
Required. Immutable. Configuration for this storage resource, which describes how it should be created or imported. This field only controls how the storage resource is initially created or imported. Subsequent changes to the storage resource should be made via the resource's API and will not be reflected in the configuration. |
Union field reference. Reference to the storage resource in Google Cloud. Exactly one of these fields will be populated based on the configured type of storage resource. reference can be only one of the following: |
|
filestore |
Reference to a Filestore instance. Populated if and only if the storage resource was configured to use Filestore. |
bucket |
Reference to a Google Cloud Storage bucket. Populated if and only if the storage resource was configured to use Google Cloud Storage. |
lustre |
Reference to a Managed Lustre instance. Populated if and only if the storage resource was configured to use Managed Lustre. |
FilestoreReference
A reference to a Filestore instance.
| JSON representation |
|---|
{ "filestore": string } |
| Fields | |
|---|---|
filestore |
Output only. Name of the Filestore instance, in the format |
BucketReference
A reference to a Google Cloud Storage bucket.
| JSON representation |
|---|
{ "bucket": string } |
| Fields | |
|---|---|
bucket |
Output only. Name of the bucket. |
LustreReference
A reference to a Managed Lustre instance.
| JSON representation |
|---|
{ "lustre": string } |
| Fields | |
|---|---|
lustre |
Output only. Name of the Managed Lustre instance, in the format |
StorageResourceConfig
Describes how a storage resource should be initialized. Each storage resource can either be imported from an existing Google Cloud resource or initialized when the cluster is created.
| JSON representation |
|---|
{ // Union field |
| Fields | |
|---|---|
Union field config. Particular type of configuration for this storage resource. config can be only one of the following: |
|
newFilestore |
Optional. Immutable. If set, indicates that a new Filestore instance should be created. |
existingFilestore |
Optional. Immutable. If set, indicates that an existing Filestore instance should be imported. |
newBucket |
Optional. Immutable. If set, indicates that a new Cloud Storage bucket should be created. |
existingBucket |
Optional. Immutable. If set, indicates that an existing Cloud Storage bucket should be imported. |
newLustre |
Optional. Immutable. If set, indicates that a new Managed Lustre instance should be created. |
existingLustre |
Optional. Immutable. If set, indicates that an existing Managed Lustre instance should be imported. |
NewFilestoreConfig
When set in a StorageResourceConfig, indicates that a new Filestore instance should be created.
| JSON representation |
|---|
{ "filestore": string, "description": string, "fileShares": [ { object ( |
| Fields | |
|---|---|
filestore |
Required. Immutable. Name of the Filestore instance to create, in the format |
description |
Optional. Immutable. Description of the instance. Maximum of 2048 characters. |
fileShares[] |
Required. Immutable. File system shares on the instance. Exactly one file share must be specified. |
tier |
Required. Immutable. Service tier to use for the instance. |
protocol |
Optional. Immutable. Access protocol to use for all file shares in the instance. Defaults to NFS V3 if not set. |
Tier
Available service tiers for Filestore instances.
| Enums | |
|---|---|
TIER_UNSPECIFIED |
Not set. |
ZONAL |
Offers expanded capacity and performance scaling capabilities suitable for high-performance computing application requirements. |
REGIONAL |
Offers features and availability needed for mission-critical, high-performance computing workloads. |
Protocol
File access protocol for Filestore instances.
| Enums | |
|---|---|
PROTOCOL_UNSPECIFIED |
Not set. |
NFSV3 |
NFS 3.0. |
NFSV41 |
NFS 4.1. |
ExistingFilestoreConfig
When set in a StorageResourceConfig, indicates that an existing Filestore instance should be imported.
| JSON representation |
|---|
{ "filestore": string } |
| Fields | |
|---|---|
filestore |
Required. Immutable. Name of the Filestore instance to import, in the format |
NewBucketConfig
When set in a StorageResourceConfig, indicates that a new Google Cloud Storage bucket should be created.
| JSON representation |
|---|
{ "bucket": string, "hierarchicalNamespace": { object ( |
| Fields | |
|---|---|
bucket |
Required. Immutable. Name of the Cloud Storage bucket to create. |
hierarchicalNamespace |
Optional. Immutable. If set, indicates that the bucket should use hierarchical namespaces. |
Union field option. Storage class of the bucket, which can be set automatically or explicitly. option can be only one of the following: |
|
autoclass |
Optional. Immutable. If set, indicates that the bucket should use Autoclass. |
storageClass |
Optional. Immutable. If set, uses the provided storage class as the bucket's default storage class. |
GcsAutoclassConfig
Message describing Google Cloud Storage autoclass configuration
| JSON representation |
|---|
{
"enabled": boolean,
"terminalStorageClass": enum ( |
| Fields | |
|---|---|
enabled |
Required. Enables Auto-class feature. |
terminalStorageClass |
Optional. Terminal storage class of the autoclass bucket |
TerminalStorageClass
Terminal storage class types of the autoclass bucket
| Enums | |
|---|---|
TERMINAL_STORAGE_CLASS_UNSPECIFIED |
Unspecified terminal storage class |
StorageClass
Storage class for a Cloud Storage bucket.
| Enums | |
|---|---|
STORAGE_CLASS_UNSPECIFIED |
Not set. |
STANDARD |
Best for data that is frequently accessed. |
NEARLINE |
Low-cost storage for data that is accessed less frequently. |
COLDLINE |
Very low-cost storage for infrequently accessed data. |
ARCHIVE |
Lowest-cost storage for data archiving, online backup, and disaster recovery. |
GcsHierarchicalNamespaceConfig
Message describing Google Cloud Storage hierarchical namespace configuration
| JSON representation |
|---|
{ "enabled": boolean } |
| Fields | |
|---|---|
enabled |
Required. Enables hierarchical namespace setup for the bucket. |
ExistingBucketConfig
When set in a StorageResourceConfig, indicates that an existing Google Cloud Storage bucket should be imported.
| JSON representation |
|---|
{ "bucket": string } |
| Fields | |
|---|---|
bucket |
Required. Immutable. Name of the Cloud Storage bucket to import. |
NewLustreConfig
When set in a StorageResourceConfig, indicates that a new Managed Lustre instance should be created.
| JSON representation |
|---|
{ "lustre": string, "description": string, "filesystem": string, "capacityGb": string } |
| Fields | |
|---|---|
lustre |
Required. Immutable. Name of the Managed Lustre instance to create, in the format |
description |
Optional. Immutable. Description of the Managed Lustre instance. Maximum of 2048 characters. |
filesystem |
Required. Immutable. Filesystem name for this instance. This name is used by client-side tools, including when mounting the instance. Must be 8 characters or less and can only contain letters and numbers. |
capacityGb |
Required. Immutable. Storage capacity of the instance in gibibytes (GiB). Allowed values are between 18000 and 7632000. |
ExistingLustreConfig
When set in a StorageResourceConfig, indicates that an existing Managed Lustre instance should be imported.
| JSON representation |
|---|
{ "lustre": string } |
| Fields | |
|---|---|
lustre |
Required. Immutable. Name of the Managed Lustre instance to import, in the format |
ComputeResource
A resource defining how virtual machines and accelerators should be provisioned for the cluster.
| JSON representation |
|---|
{
"config": {
object ( |
| Fields | |
|---|---|
config |
Required. Immutable. Configuration for this compute resource, which describes how it should be created at runtime. |
ComputeResourceConfig
Describes how a compute resource should be created at runtime.
| JSON representation |
|---|
{ // Union field |
| Fields | |
|---|---|
Union field config. Particular type of configuration for this compute resource. config can be only one of the following: |
|
newOnDemandInstances |
Optional. Immutable. If set, indicates that this resource should use on-demand VMs. |
newSpotInstances |
Optional. Immutable. If set, indicates that this resource should use spot VMs. |
newReservedInstances |
Optional. Immutable. If set, indicates that this resource should use reserved VMs. |
newFlexStartInstances |
Optional. Immutable. If set, indicates that this resource should use flex-start VMs. |
NewOnDemandInstancesConfig
When set in a ComputeResourceConfig, indicates that on-demand (i.e., using the standard provisioning model) VM instances should be created.
| JSON representation |
|---|
{ "zone": string, "machineType": string } |
| Fields | |
|---|---|
zone |
Required. Immutable. Name of the zone in which VM instances should run, e.g., |
machineType |
Required. Immutable. Name of the Compute Engine machine type to use, e.g. |
NewSpotInstancesConfig
When set in a ComputeResourceConfig, indicates that spot VM instances should be created.
| JSON representation |
|---|
{ "zone": string, "machineType": string } |
| Fields | |
|---|---|
zone |
Required. Immutable. Name of the zone in which VM instances should run, e.g., |
machineType |
Required. Immutable. Name of the Compute Engine machine type to use, e.g. |
NewReservedInstancesConfig
When set in a ComputeResourceConfig, indicates that VM instances should be created from a reservation.
| JSON representation |
|---|
{ // Union field |
| Fields | |
|---|---|
Union field source. Source of the reservation source can be only one of the following: |
|
reservation |
Optional. Immutable. Name of the reservation from which VM instances should be created, in the format |
NewFlexStartInstancesConfig
When set in a ComputeResourceConfig, indicates that VM instances should be created using Flex Start.
| JSON representation |
|---|
{ "zone": string, "machineType": string, "maxDuration": string } |
| Fields | |
|---|---|
zone |
Required. Immutable. Name of the zone in which VM instances should run, e.g., |
machineType |
Required. Immutable. Name of the Compute Engine machine type to use, e.g. |
maxDuration |
Required. Immutable. Specifies the time limit for created instances. Instances will be terminated at the end of this duration. A duration in seconds with up to nine fractional digits, ending with ' |
Orchestrator
The component responsible for scheduling and running workloads on the cluster as well as providing the user interface for interacting with the cluster at runtime.
| JSON representation |
|---|
{ // Union field |
| Fields | |
|---|---|
Union field option. Particular type of orchestrator to use in this cluster. option can be only one of the following: |
|
slurm |
Optional. If set, indicates that the cluster should use Slurm as the orchestrator. |
SlurmOrchestrator
When set in Orchestrator, indicates that the cluster should use Slurm as the orchestrator.
| JSON representation |
|---|
{ "loginNodes": { object ( |
| Fields | |
|---|---|
loginNodes |
Required. Configuration for login nodes, which allow users to access the cluster over SSH. |
nodeSets[] |
Required. Configuration of Slurm nodesets, which define groups of compute resources that can be used by Slurm. At least one compute node is required. |
partitions[] |
Required. Configuration of Slurm partitions, which group one or more nodesets. Acts as a queue against which jobs can be submitted. At least one partition is required. |
defaultPartition |
Optional. Default partition to use for submitted jobs that do not explicitly specify a partition. Required if and only if there is more than one partition, in which case it must match the id of one of the partitions. |
prologBashScripts[] |
Optional. Slurm prolog scripts, which will be executed by compute nodes before a node begins running a new job. Values must not be empty. |
epilogBashScripts[] |
Optional. Slurm epilog scripts, which will be executed by compute nodes whenever a node finishes running a job. Values must not be empty. |
SlurmLoginNodes
Configuration for Slurm login nodes in the cluster. Login nodes are Compute Engine VM instances that allow users to access the cluster over SSH.
| JSON representation |
|---|
{ "count": string, "zone": string, "machineType": string, "startupScript": string, "enableOsLogin": boolean, "enablePublicIps": boolean, "labels": { string: string, ... }, "storageConfigs": [ { object ( |
| Fields | |
|---|---|
count |
Required. Number of login node instances to create. |
zone |
Required. Name of the zone in which login nodes should run, e.g., |
machineType |
Required. Name of the Compute Engine machine type to use for login nodes, e.g. |
startupScript |
Optional. Startup script to be run on each login node instance. Max 256KB. |
enableOsLogin |
Optional. Whether OS Login should be enabled on login node instances. |
enablePublicIps |
Optional. Whether login node instances should be assigned external IP addresses. |
labels |
Optional. Labels that should be applied to each login node instance. An object containing a list of |
storageConfigs[] |
Optional. How |
instances[] |
Output only. Information about the login node instances that were created in Compute Engine. |
bootDisk |
Optional. Boot disk for the login node. |
StorageConfig
Description of how a storage resource should be mounted on a VM instance.
| JSON representation |
|---|
{ "id": string, "localMount": string } |
| Fields | |
|---|---|
id |
Required. ID of the storage resource to mount, which must match a key in the cluster's storageResources. |
localMount |
Required. A directory inside the VM instance's file system where the storage resource should be mounted (e.g., |
ComputeInstance
Details about a Compute Engine instance.
| JSON representation |
|---|
{ "instance": string } |
| Fields | |
|---|---|
instance |
Output only. Name of the VM instance, in the format |
BootDisk
A Persistent disk used as the boot disk for a Compute Engine VM instance.
| JSON representation |
|---|
{ "type": string, "sizeGb": string } |
| Fields | |
|---|---|
type |
Required. Immutable. Persistent disk type, in the format |
sizeGb |
Required. Immutable. Size of the disk in gigabytes. Must be at least 10GB. |
SlurmNodeSet
Configuration for Slurm nodesets in the cluster. Nodesets are groups of compute nodes used by Slurm that are responsible for running workloads submitted to the cluster.
| JSON representation |
|---|
{ "id": string, "computeId": string, "storageConfigs": [ { object ( |
| Fields | |
|---|---|
id |
Required. Identifier for the nodeset, which allows it to be referenced by partitions. Must conform to RFC-1034 (lower-case, alphanumeric, and at most 63 characters). |
computeId |
Optional. ID of the compute resource on which this nodeset will run. Must match a key in the cluster's computeResources. |
storageConfigs[] |
Optional. How |
staticNodeCount |
Optional. Number of nodes to be statically created for this nodeset. The cluster will attempt to ensure that at least this many nodes exist at all times. |
maxDynamicNodeCount |
Optional. Controls how many additional nodes a cluster can bring online to handle workloads. Set this value to enable dynamic node creation and limit the number of additional nodes the cluster can bring online. Leave empty if you do not want the cluster to create nodes dynamically, and instead rely only on static nodes. |
Union field type. Additional configuration for the nodeset. If not set, the nodeset will use ComputeInstanceSlurmNodeSet with default values. type can be only one of the following: |
|
computeInstance |
Optional. If set, indicates that the nodeset should be backed by Compute Engine instances. |
ComputeInstanceSlurmNodeSet
When set in a SlurmNodeSet, indicates that the nodeset should be backed by Compute Engine VM instances.
| JSON representation |
|---|
{
"startupScript": string,
"labels": {
string: string,
...
},
"bootDisk": {
object ( |
| Fields | |
|---|---|
startupScript |
Optional. Startup script to be run on each VM instance in the nodeset. Max 256KB. |
labels |
Optional. Labels that should be applied to each VM instance in the nodeset. An object containing a list of |
bootDisk |
Optional. Boot disk for the compute instance |
SlurmPartition
Configuration for Slurm partitions in the cluster. Partitions are groups of nodesets, and are how clients specify where their workloads should be run.
| JSON representation |
|---|
{ "id": string, "nodeSetIds": [ string ] } |
| Fields | |
|---|---|
id |
Required. ID of the partition, which is how users will identify it. Must conform to RFC-1034 (lower-case, alphanumeric, and at most 63 characters). |
nodeSetIds[] |
Required. IDs of the nodesets that make up this partition. Values must match |
Methods |
|
|---|---|
|
Creates a new Cluster in a given project and location. |
|
Deletes a single Cluster. |
|
Gets details of a single Cluster. |
|
Lists Clusters in a given project and location. |
|
Updates the parameters of a single Cluster. |