Organiser des ressources à l'aide d'étiquettes

Ce document explique comment utiliser des libellés pour organiser vos ressources Batch.

Les libellés sont des paires clé-valeur appliquées aux ressources pour les regrouper et les décrire. Batch comporte des libellés prédéfinis, qui sont automatiquement appliqués aux ressources, et des libellés personnalisés, que vous pouvez définir et appliquer lors de la création d'un job.

Les libellés vous permettent de filtrer les résultats des listes de ressources et des rapports Cloud Billing. Par exemple, vous pouvez utiliser des libellés pour effectuer les opérations suivantes :

  • Clarifier et organiser la liste des jobs de votre projet.

  • Distinguer les exécutables d'un job en utilisant des libellés pour décrire le type de conteneur ou de script qu'ils spécifient.

  • Analyser les coûts en filtrant les rapports Cloud Billing pour les ressources créées par Batch ou des jobs spécifiques.

Pour en savoir plus sur les libellés, consultez également la documentation Compute Engine sur les libellés.

Avant de commencer

  1. Si vous n'avez jamais utilisé Batch, consultez Premiers pas avec Batch et activez Batch en remplissant les prérequis pour les projets et les utilisateurs.

  2. Pour obtenir les autorisations nécessaires pour créer un job, demandez à votre administrateur de vous accorder les rôles IAM suivants :

    Pour en savoir plus sur l'attribution de rôles, consultez Gérer l'accès aux projets, aux dossiers et aux organisations.

    Vous pouvez également obtenir les autorisations requises avec des rôles personnalisés ou d'autres rôles prédéfinis.

Restrictions

En plus des exigences concernant les libellés spécifiées dans la documentation Compute Engine, l'application de libellés à un job Batch et à ses ressources présente les restrictions suivantes :

  • Batch n'est compatible qu'avec les libellés pour les ressources créées à l'aide de Batch et des types suivants :

  • Après avoir pris en compte les libellés prédéfinis que Batch applique automatiquement à un job, vous pouvez définir les quantités suivantes de libellés personnalisés :

    • Vous pouvez définir un maximum de 63 libellés personnalisés à appliquer au job et à ses exécutables.

    • Vous pouvez définir un maximum de 61 libellés personnalisés à appliquer à chaque GPU, disque persistant et VM créés pour le job.

  • Batch n'est compatible qu'avec la définition de libellés personnalisés avec des noms uniques. Cela a les conséquences suivantes :

    • Toute tentative de remplacement d'un libellé prédéfini entraîne des erreurs.

    • La définition d'un libellé personnalisé en double remplace le libellé personnalisé existant.

  • Batch n'est compatible qu'avec la définition de libellés lors de la création d'un job.

    • Il est impossible d'ajouter, de mettre à jour ou de supprimer des libellés pour les jobs et les exécutables.

    • Bien qu'il soit possible d'utiliser Compute Engine pour ajouter, mettre à jour ou supprimer des libellés pour les disques persistants et les VM créés pour les jobs, cela n'est pas recommandé. Le délai d'existence des ressources d'un job ne peut pas être estimé de manière fiable, et toute modification risque de ne pas fonctionner correctement avec Batch.

  • Pour utiliser des libellés afin de filtrer votre liste de jobs, vous devez afficher votre liste de jobs à l'aide de gcloud CLI ou de l'API Batch.

Libellés prédéfinis

Chaque libellé prédéfini comporte une clé qui commence par le préfixe batch-. Par défaut, Batch applique automatiquement les libellés prédéfinis suivants :

  • À chaque job que vous créez :

    • batch-job-id : la valeur de ce libellé est définie sur le nom du job.

  • À chaque GPU, disque persistant et VM créés pour un job :

    • batch-job-id : la valeur de ce libellé est définie sur le nom du job.

    • batch-job-uid: la valeur de ce libellé est définie sur l'identifiant unique (UID) du job.

    • batch-node: la valeur de ce libellé est nulle. Il regroupe simplement tous les GPU, disques persistants et VM créés pour les jobs. Par exemple, utilisez ce libellé lorsque vous consultez un rapport Cloud Billing pour identifier les coûts de tous les GPU, disques persistants et VM créés par Batch.

Définir des libellés personnalisés

Vous pouvez éventuellement définir un ou plusieurs libellés personnalisés lors de la création d'un job. Vous pouvez définir des libellés personnalisés avec de nouvelles clés ou des clés que votre projet utilise déjà. Pour définir des libellés personnalisés, sélectionnez une ou plusieurs des méthodes suivantes dans ce document en fonction de l'objectif du libellé :

  • Définir des libellés personnalisés pour le job et ses ressources.

    Cette section explique comment appliquer un ou plusieurs libellés personnalisés au job et à chaque GPU, disque persistant et VM créés pour le job. Après avoir créé le job, vous pouvez utiliser ces libellés pour filtrer les rapports Cloud Billing et les listes de jobs, de disques persistants et de VM de votre projet.

  • Définir des libellés personnalisés pour le job.

    Cette section explique comment appliquer un ou plusieurs libellés personnalisés au job. Après avoir créé le job, vous pouvez utiliser ces libellés pour filtrer les listes de jobs de votre projet.

  • Définir des libellés personnalisés pour les exécutables.

    Cette section explique comment appliquer un ou plusieurs libellés personnalisés à un ou plusieurs exécutables pour le job. Après avoir créé le job, vous pouvez utiliser ces libellés pour filtrer les listes de jobs de votre projet.

Définir des libellés personnalisés pour le job et ses ressources

Les libellés définis dans le labels champ pour la stratégie d'allocation d'un job sont appliqués au job, ainsi qu'à chaque GPU (le cas échéant), disque persistant (tous les disques de démarrage et tous les nouveaux volumes de stockage) et VM créés pour le job.

Vous pouvez définir des libellés pour un job et ses ressources lorsque vous créez un job à l'aide de gcloud CLI ou de l'API Batch.

gcloud

Par exemple, pour créer un job de conteneur de base dans us-central1 qui définit deux libellés personnalisés s'appliquant au job et aux ressources créées pour le job, procédez comme suit :

  1. Créez un fichier JSON qui spécifie les détails de configuration du job et le allocationPolicy.labels champ.

    {
  "allocationPolicy": {
    "instances": [
      {
        "policy": {
          "machineType": "e2-standard-4"
        }
      }
    ],
    "labels": {
      "VM_LABEL_NAME1": "VM_LABEL_VALUE1",
      "VM_LABEL_NAME2": "VM_LABEL_VALUE2"
    }
  },
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "container": {
              "imageUri": "gcr.io/google-containers/busybox",
              "entrypoint": "/bin/sh",
              "commands": [
                "-c",
                "echo Hello world!"
              ]
            }
          }
        ]
      }
    }
  ]
}

    Remplacez les éléments suivants :

    • VM_LABEL_NAME1: nom du premier libellé à appliquer aux VM créées pour le job.

    • VM_LABEL_VALUE1: valeur du premier libellé à appliquer aux VM créées pour le job.

    • VM_LABEL_NAME2: nom du deuxième libellé à appliquer aux VM créées pour le job.

    • VM_LABEL_VALUE2: valeur du deuxième libellé à appliquer aux VM créées pour le job.

  2. Créez le job dans us-central1 à l'aide de la gcloud batch jobs submit commande.

    gcloud batch jobs submit example-job \
    --config=JSON_CONFIGURATION_FILE \
    --location=us-central1

    Remplacez JSON_CONFIGURATION_FILE par le chemin d'accès au fichier JSON contenant les détails de configuration du job que vous avez créé à l'étape précédente.

API

Par exemple, pour créer un job de conteneur de base dans us-central1 qui définit deux libellés personnalisés s'appliquant au job et aux ressources créées pour le job, envoyez une requête POST à la jobs.create méthode et spécifiez le allocationPolicy.labels champ.

POST https://batch.googleapis.com/v1/projects/example-project/locations/us-central1/jobs?job_id=example-job

{
  "allocationPolicy": {
    "instances": [
      {
        "policy": {
          "machineType": "e2-standard-4"
        }
      }
    ],
    "labels": {
      "VM_LABEL_NAME1": "VM_LABEL_VALUE1",
      "VM_LABEL_NAME2": "VM_LABEL_VALUE2"
    }
  },
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "container": {
              "imageUri": "gcr.io/google-containers/busybox",
              "entrypoint": "/bin/sh",
              "commands": [
                "-c",
                "echo Hello world!"
              ]
            }
          }
        ]
      }
    }
  ]
}

Remplacez les éléments suivants :

  • VM_LABEL_NAME1: nom du premier libellé à appliquer aux VM créées pour le job.

  • VM_LABEL_VALUE1: valeur du premier libellé à appliquer aux VM créées pour le job.

  • VM_LABEL_NAME2: nom du deuxième libellé à appliquer aux VM créées pour le job.

  • VM_LABEL_VALUE2: valeur du deuxième libellé à appliquer aux VM créées pour le job.

Java 


import com.google.cloud.batch.v1.AllocationPolicy;
import com.google.cloud.batch.v1.BatchServiceClient;
import com.google.cloud.batch.v1.ComputeResource;
import com.google.cloud.batch.v1.CreateJobRequest;
import com.google.cloud.batch.v1.Job;
import com.google.cloud.batch.v1.LogsPolicy;
import com.google.cloud.batch.v1.Runnable;
import com.google.cloud.batch.v1.TaskGroup;
import com.google.cloud.batch.v1.TaskSpec;
import com.google.protobuf.Duration;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

public class CreateBatchAllocationPolicyLabel {

  public static void main(String[] args)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // TODO(developer): Replace these variables before running the sample.
    // Project ID or project number of the Google Cloud project you want to use.
    String projectId = "YOUR_PROJECT_ID";
    // Name of the region you want to use to run the job. Regions that are
    // available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
    String region = "us-central1";
    // The name of the job that will be created.
    // It needs to be unique for each project and region pair.
    String jobName = "example-job";
    // Name of the label1 to be applied for your Job.
    String labelName1 = "VM_LABEL_NAME1";
    // Value for the label1 to be applied for your Job.
    String labelValue1 = "VM_LABEL_VALUE1";
    // Name of the label2 to be applied for your Job.
    String labelName2 = "VM_LABEL_NAME2";
    // Value for the label2 to be applied for your Job.
    String labelValue2 = "VM_LABEL_VALUE2";

    createBatchAllocationPolicyLabel(projectId, region, jobName, labelName1,
        labelValue1, labelName2, labelValue2);
  }

  // This method shows how to create a job with labels defined 
  // in the labels field of a job's allocation policy. These are 
  // applied to the job, as well as to each GPU (if any), persistent disk 
  // (all boot disks and any new storage volumes), and VM created for the job.
  public static Job createBatchAllocationPolicyLabel(String projectId, String region,
                               String jobName, String labelName1,
                               String labelValue1, String labelName2, String labelValue2)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests.
    try (BatchServiceClient batchServiceClient = BatchServiceClient.create()) {

      // Define what will be done as part of the job.
      Runnable runnable =
          Runnable.newBuilder()
              .setContainer(
                  Runnable.Container.newBuilder()
                      .setImageUri("gcr.io/google-containers/busybox")
                      .setEntrypoint("/bin/sh")
                      .addCommands("-c")
                      .addCommands(
                          "echo Hello world! This is task ${BATCH_TASK_INDEX}. "
                              + "This job has a total of ${BATCH_TASK_COUNT} tasks.")
                      .build())
              .build();

      // We can specify what resources are requested by each task.
      ComputeResource computeResource =
          ComputeResource.newBuilder()
              // In milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
              .setCpuMilli(2000)
              // In MiB.
              .setMemoryMib(2000)
              .build();

      TaskSpec task =
          TaskSpec.newBuilder()
              // Jobs can be divided into tasks. In this case, we have only one task.
              .addRunnables(runnable)
              .setComputeResource(computeResource)
              .setMaxRetryCount(2)
              .setMaxRunDuration(Duration.newBuilder().setSeconds(3600).build())
              .build();

      // Tasks are grouped inside a job using TaskGroups.
      // Currently, it's possible to have only one task group.
      TaskGroup taskGroup = TaskGroup.newBuilder().setTaskCount(1).setTaskSpec(task).build();

      // Policies are used to define on what kind of virtual machines the tasks will run on.
      // In this case, we tell the system to use "e2-standard-4" machine type.
      // Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
      AllocationPolicy.InstancePolicy instancePolicy =
          AllocationPolicy.InstancePolicy.newBuilder().setMachineType("e2-standard-4").build();

      AllocationPolicy allocationPolicy =
          AllocationPolicy.newBuilder()
              .addInstances(AllocationPolicy.InstancePolicyOrTemplate.newBuilder()
                  .setPolicy(instancePolicy)
                  .build())
              // Labels and their value to be applied to the job and its resources
              .putLabels(labelName1, labelValue1)
              .putLabels(labelName2, labelValue2)
              .build();

      Job job =
          Job.newBuilder()
              .addTaskGroups(taskGroup)
              .setAllocationPolicy(allocationPolicy)
              // We use Cloud Logging as it's an out of the box available option.
              .setLogsPolicy(LogsPolicy.newBuilder()
                      .setDestination(LogsPolicy.Destination.CLOUD_LOGGING).build())
              .build();

      CreateJobRequest createJobRequest =
          CreateJobRequest.newBuilder()
              // The job's parent is the region in which the job will run.
              .setParent(String.format("projects/%s/locations/%s", projectId, region))
              .setJob(job)
              .setJobId(jobName)
              .build();

      Job result =
          batchServiceClient
              .createJobCallable()
              .futureCall(createJobRequest)
              .get(5, TimeUnit.MINUTES);

      System.out.printf("Successfully created the job: %s", result.getName());

      return result;
    }
  }

}

Node.js 

// Imports the Batch library
const batchLib = require('@google-cloud/batch');
const batch = batchLib.protos.google.cloud.batch.v1;

// Instantiates a client
const batchClient = new batchLib.v1.BatchServiceClient();

/**
 * TODO(developer): Update these variables before running the sample.
 */
// Project ID or project number of the Google Cloud project you want to use.
const projectId = await batchClient.getProjectId();
// Name of the region you want to use to run the job. Regions that are
// available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
const region = 'europe-central2';
// The name of the job that will be created.
// It needs to be unique for each project and region pair.
const jobName = 'batch-labels-allocation-job';
// Name of the label1 to be applied for your Job.
const labelName1 = 'vm_label_name_1';
// Value for the label1 to be applied for your Job.
const labelValue1 = 'vmLabelValue1';
// Name of the label2 to be applied for your Job.
const labelName2 = 'vm_label_name_2';
// Value for the label2 to be applied for your Job.
const labelValue2 = 'vmLabelValue2';

// Define what will be done as part of the job.
const runnable = new batch.Runnable({
  script: new batch.Runnable.Script({
    commands: ['-c', 'echo Hello world! This is task ${BATCH_TASK_INDEX}.'],
  }),
});

// Specify what resources are requested by each task.
const computeResource = new batch.ComputeResource({
  // In milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
  cpuMilli: 500,
  // In MiB.
  memoryMib: 16,
});

const task = new batch.TaskSpec({
  runnables: [runnable],
  computeResource,
  maxRetryCount: 2,
  maxRunDuration: {seconds: 3600},
});

// Tasks are grouped inside a job using TaskGroups.
const group = new batch.TaskGroup({
  taskCount: 3,
  taskSpec: task,
});

// Policies are used to define on what kind of virtual machines the tasks will run on.
// In this case, we tell the system to use "e2-standard-4" machine type.
// Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
const instancePolicy = new batch.AllocationPolicy.InstancePolicy({
  machineType: 'e2-standard-4',
});

const allocationPolicy = new batch.AllocationPolicy({
  instances: [{policy: instancePolicy}],
});
// Labels and their value to be applied to the job and its resources.
allocationPolicy.labels[labelName1] = labelValue1;
allocationPolicy.labels[labelName2] = labelValue2;

const job = new batch.Job({
  name: jobName,
  taskGroups: [group],
  labels: {env: 'testing', type: 'script'},
  allocationPolicy,
  // We use Cloud Logging as it's an option available out of the box
  logsPolicy: new batch.LogsPolicy({
    destination: batch.LogsPolicy.Destination.CLOUD_LOGGING,
  }),
});

// The job's parent is the project and region in which the job will run
const parent = `projects/${projectId}/locations/${region}`;

async function callCreateBatchLabelsAllocation() {
  // Construct request
  const request = {
    parent,
    jobId: jobName,
    job,
  };

  // Run request
  const [response] = await batchClient.createJob(request);
  console.log(JSON.stringify(response));
}

await callCreateBatchLabelsAllocation();

Python 

from google.cloud import batch_v1


def create_job_with_custom_allocation_policy_labels(
    project_id: str, region: str, job_name: str, labels: dict
) -> batch_v1.Job:
    """
    This method shows the creation of a Batch job with custom labels which describe the allocation policy.
    Args:
        project_id (str): project ID or project number of the Cloud project you want to use.
        region (str): name of the region you want to use to run the job. Regions that are
            available for Batch are listed on: https://cloud.google.com/batch/docs/locations
        job_name (str): the name of the job that will be created.
        labels (dict): a dictionary of key-value pairs that will be used as labels
            E.g., {"label_key1": "label_value2", "label_key2": "label_value2"}
    Returns:
        batch_v1.Job: The created Batch job object containing configuration details.
    """
    client = batch_v1.BatchServiceClient()

    runnable = batch_v1.Runnable()
    runnable.container = batch_v1.Runnable.Container()
    runnable.container.image_uri = "gcr.io/google-containers/busybox"
    runnable.container.entrypoint = "/bin/sh"
    runnable.container.commands = [
        "-c",
        "echo Hello world!",
    ]

    # Create a task specification and assign the runnable and volume to it
    task = batch_v1.TaskSpec()
    task.runnables = [runnable]

    # Specify what resources are requested by each task.
    resources = batch_v1.ComputeResource()
    resources.cpu_milli = 2000  # in milliseconds per cpu-second. This means the task requires 2 whole CPUs.
    resources.memory_mib = 16  # in MiB
    task.compute_resource = resources

    task.max_retry_count = 2
    task.max_run_duration = "3600s"

    # Create a task group and assign the task specification to it
    group = batch_v1.TaskGroup()
    group.task_count = 3
    group.task_spec = task

    # Policies are used to define on what kind of virtual machines the tasks will run on.
    # In this case, we tell the system to use "e2-standard-4" machine type.
    # Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
    policy = batch_v1.AllocationPolicy.InstancePolicy()
    policy.machine_type = "e2-standard-4"
    instances = batch_v1.AllocationPolicy.InstancePolicyOrTemplate()
    instances.policy = policy
    allocation_policy = batch_v1.AllocationPolicy()
    allocation_policy.instances = [instances]

    # Assign the provided labels to the allocation policy
    allocation_policy.labels = labels

    # Create the job and assign the task group and allocation policy to it
    job = batch_v1.Job()
    job.task_groups = [group]
    job.allocation_policy = allocation_policy

    # We use Cloud Logging as it's an out of the box available option
    job.logs_policy = batch_v1.LogsPolicy()
    job.logs_policy.destination = batch_v1.LogsPolicy.Destination.CLOUD_LOGGING

    # Create the job request and set the job and job ID
    create_request = batch_v1.CreateJobRequest()
    create_request.job = job
    create_request.job_id = job_name
    # The job's parent is the region in which the job will run
    create_request.parent = f"projects/{project_id}/locations/{region}"

    return client.create_job(create_request)

Définir des libellés personnalisés pour le job

Les libellés définis dans le labels champ pour le job ne sont appliqués qu'au job.

Vous pouvez définir des libellés pour un job lorsque vous créez un job à l'aide de gcloud CLI ou de l'API Batch.

gcloud

Par exemple, pour créer un job de conteneur de base dans us-central1 qui définit deux libellés personnalisés s'appliquant au job lui-même, procédez comme suit :

  1. Créez un fichier JSON qui spécifie les détails de configuration du job et le labels champ.

    {
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "container": {
              "imageUri": "gcr.io/google-containers/busybox",
              "entrypoint": "/bin/sh",
              "commands": [
                "-c",
                "echo Hello World!"
              ]
            }
          }
        ]
      }
    }
  ],
  "labels": {
    "JOB_LABEL_NAME1": "JOB_LABEL_VALUE1",
    "JOB_LABEL_NAME2": "JOB_LABEL_VALUE2"
  }
}

    Remplacez les éléments suivants :

    • JOB_LABEL_NAME1: nom du premier libellé à appliquer à votre job.

    • JOB_LABEL_VALUE1: valeur du premier libellé à appliquer à votre job.

    • JOB_LABEL_NAME2: nom du deuxième libellé à appliquer à votre job.

    • JOB_LABEL_VALUE2: valeur du deuxième libellé à appliquer à votre job.

  2. Créez le job dans us-central1 à l'aide de la gcloud batch jobs submit commande avec les indicateurs suivants :

    gcloud batch jobs submit example-job \
    --config=JSON_CONFIGURATION_FILE \
    --location=us-central1

    Remplacez JSON_CONFIGURATION_FILE par le chemin d'accès au fichier JSON contenant les détails de configuration du job que vous avez créé à l'étape précédente.

API

Par exemple, pour créer un job de conteneur dans us-central1 qui définit deux libellés personnalisés à appliquer au job lui-même, envoyez une requête POST à la jobs.create méthode et spécifiez le labels champ.

POST https://batch.googleapis.com/v1/projects/example-project/locations/us-central1/jobs?job_id=example-job

{
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "container": {
              "imageUri": "gcr.io/google-containers/busybox",
              "entrypoint": "/bin/sh",
              "commands": [
                "-c",
                "echo Hello World!"
              ]
            }
          }
        ]
      }
    }
  ],
  "labels": {
    "JOB_LABEL_NAME1": "JOB_LABEL_VALUE1",
    "JOB_LABEL_NAME2": "JOB_LABEL_VALUE2"
  }
}

Remplacez les éléments suivants :

  • JOB_LABEL_NAME1: nom du premier libellé à appliquer à votre job.

  • JOB_LABEL_VALUE1: valeur du premier libellé à appliquer à votre job.

  • JOB_LABEL_NAME2: nom du deuxième libellé à appliquer à votre job.

  • JOB_LABEL_VALUE2: valeur du deuxième libellé à appliquer à votre job.

Java 


import com.google.cloud.batch.v1.BatchServiceClient;
import com.google.cloud.batch.v1.ComputeResource;
import com.google.cloud.batch.v1.CreateJobRequest;
import com.google.cloud.batch.v1.Job;
import com.google.cloud.batch.v1.LogsPolicy;
import com.google.cloud.batch.v1.Runnable;
import com.google.cloud.batch.v1.TaskGroup;
import com.google.cloud.batch.v1.TaskSpec;
import com.google.protobuf.Duration;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;


public class CreateBatchLabelJob {

  public static void main(String[] args)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // TODO(developer): Replace these variables before running the sample.
    // Project ID or project number of the Google Cloud project you want to use.
    String projectId = "YOUR_PROJECT_ID";
    // Name of the region you want to use to run the job. Regions that are
    // available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
    String region = "us-central1";
    // The name of the job that will be created.
    // It needs to be unique for each project and region pair.
    String jobName = "example-job";
    // Name of the label1 to be applied for your Job.
    String labelName1 = "JOB_LABEL_NAME1";
    // Value for the label1 to be applied for your Job.
    String labelValue1 = "JOB_LABEL_VALUE1";
    // Name of the label2 to be applied for your Job.
    String labelName2 = "JOB_LABEL_NAME2";
    // Value for the label2 to be applied for your Job.
    String labelValue2 = "JOB_LABEL_VALUE2";

    createBatchLabelJob(projectId, region, jobName, labelName1,
        labelValue1, labelName2, labelValue2);
  }

  // Creates a job with labels defined in the labels field.
  public static Job createBatchLabelJob(String projectId, String region, String jobName,
                    String labelName1, String labelValue1, String labelName2, String labelValue2)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests.
    try (BatchServiceClient batchServiceClient = BatchServiceClient.create()) {

      // Define what will be done as part of the job.
      Runnable runnable =
          Runnable.newBuilder()
              .setContainer(
                  Runnable.Container.newBuilder()
                      .setImageUri("gcr.io/google-containers/busybox")
                      .setEntrypoint("/bin/sh")
                      .addCommands("-c")
                      .addCommands(
                          "echo Hello world! This is task ${BATCH_TASK_INDEX}. "
                              + "This job has a total of ${BATCH_TASK_COUNT} tasks.")
                      .build())
              .build();

      // We can specify what resources are requested by each task.
      ComputeResource computeResource =
          ComputeResource.newBuilder()
              // In milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
              .setCpuMilli(2000)
              // In MiB.
              .setMemoryMib(2000)
              .build();

      TaskSpec task =
          TaskSpec.newBuilder()
              // Jobs can be divided into tasks. In this case, we have only one task.
              .addRunnables(runnable)
              .setComputeResource(computeResource)
              .setMaxRetryCount(2)
              .setMaxRunDuration(Duration.newBuilder().setSeconds(3600).build())
              .build();

      // Tasks are grouped inside a job using TaskGroups.
      // Currently, it's possible to have only one task group.
      TaskGroup taskGroup = TaskGroup.newBuilder().setTaskCount(1).setTaskSpec(task).build();

      Job job =
          Job.newBuilder()
              .addTaskGroups(taskGroup)
              // We use Cloud Logging as it's an out of the box available option.
              .setLogsPolicy(LogsPolicy.newBuilder()
              .setDestination(LogsPolicy.Destination.CLOUD_LOGGING).build())
              // Labels and their value to be applied to the job.
              .putLabels(labelName1, labelValue1)
              .putLabels(labelName2, labelValue2)
              .build();

      CreateJobRequest createJobRequest =
          CreateJobRequest.newBuilder()
              // The job's parent is the region in which the job will run.
              .setParent(String.format("projects/%s/locations/%s", projectId, region))
              .setJob(job)
              .setJobId(jobName)
              .build();

      Job result =
          batchServiceClient
              .createJobCallable()
              .futureCall(createJobRequest)
              .get(5, TimeUnit.MINUTES);

      System.out.printf("Successfully created the job: %s", result.getName());

      return result;
    }
  }

}

Node.js 

// Imports the Batch library
const batchLib = require('@google-cloud/batch');
const batch = batchLib.protos.google.cloud.batch.v1;

// Instantiates a client
const batchClient = new batchLib.v1.BatchServiceClient();

/**
 * TODO(developer): Update these variables before running the sample.
 */
// Project ID or project number of the Google Cloud project you want to use.
const projectId = await batchClient.getProjectId();
// Name of the region you want to use to run the job. Regions that are
// available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
const region = 'europe-central2';
// The name of the job that will be created.
// It needs to be unique for each project and region pair.
const jobName = 'batch-labels-job';
// Name of the label1 to be applied for your Job.
const labelName1 = 'job_label_name_1';
// Value for the label1 to be applied for your Job.
const labelValue1 = 'job_label_value1';
// Name of the label2 to be applied for your Job.
const labelName2 = 'job_label_name_2';
// Value for the label2 to be applied for your Job.
const labelValue2 = 'job_label_value2';

// Define what will be done as part of the job.
const runnable = new batch.Runnable({
  container: new batch.Runnable.Container({
    imageUri: 'gcr.io/google-containers/busybox',
    entrypoint: '/bin/sh',
    commands: ['-c', 'echo Hello world! This is task ${BATCH_TASK_INDEX}.'],
  }),
});

// Specify what resources are requested by each task.
const computeResource = new batch.ComputeResource({
  // In milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
  cpuMilli: 500,
  // In MiB.
  memoryMib: 16,
});

const task = new batch.TaskSpec({
  runnables: [runnable],
  computeResource,
  maxRetryCount: 2,
  maxRunDuration: {seconds: 3600},
});

// Tasks are grouped inside a job using TaskGroups.
const group = new batch.TaskGroup({
  taskCount: 3,
  taskSpec: task,
});

const job = new batch.Job({
  name: jobName,
  taskGroups: [group],
  // We use Cloud Logging as it's an option available out of the box
  logsPolicy: new batch.LogsPolicy({
    destination: batch.LogsPolicy.Destination.CLOUD_LOGGING,
  }),
});

// Labels and their value to be applied to the job and its resources.
job.labels[labelName1] = labelValue1;
job.labels[labelName2] = labelValue2;

// The job's parent is the project and region in which the job will run
const parent = `projects/${projectId}/locations/${region}`;

async function callCreateBatchLabelsJob() {
  // Construct request
  const request = {
    parent,
    jobId: jobName,
    job,
  };

  // Run request
  const [response] = await batchClient.createJob(request);
  console.log(JSON.stringify(response));
}

await callCreateBatchLabelsJob();

Python 

from google.cloud import batch_v1


def create_job_with_custom_job_labels(
    project_id: str,
    region: str,
    job_name: str,
    labels: dict,
) -> batch_v1.Job:
    """
    This method creates a Batch job with custom labels.
    Args:
        project_id (str): project ID or project number of the Cloud project you want to use.
        region (str): name of the region you want to use to run the job. Regions that are
            available for Batch are listed on: https://cloud.google.com/batch/docs/locations
        job_name (str): the name of the job that will be created.
        labels (dict): A dictionary of custom labels to be added to the job.
            E.g., {"label_key1": "label_value2", "label_key2": "label_value2"}
    Returns:
        batch_v1.Job: The created Batch job object containing configuration details.
    """
    client = batch_v1.BatchServiceClient()

    runnable = batch_v1.Runnable()
    runnable.container = batch_v1.Runnable.Container()
    runnable.container.image_uri = "gcr.io/google-containers/busybox"
    runnable.container.entrypoint = "/bin/sh"
    runnable.container.commands = [
        "-c",
        "echo Hello world!",
    ]

    # Create a task specification and assign the runnable and volume to it
    task = batch_v1.TaskSpec()
    task.runnables = [runnable]

    # Specify what resources are requested by each task.
    resources = batch_v1.ComputeResource()
    resources.cpu_milli = 2000  # in milliseconds per cpu-second. This means the task requires 2 whole CPUs.
    resources.memory_mib = 16  # in MiB
    task.compute_resource = resources

    task.max_retry_count = 2
    task.max_run_duration = "3600s"

    # Create a task group and assign the task specification to it
    group = batch_v1.TaskGroup()
    group.task_count = 3
    group.task_spec = task

    # Policies are used to define on what kind of virtual machines the tasks will run on.
    # In this case, we tell the system to use "e2-standard-4" machine type.
    # Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
    policy = batch_v1.AllocationPolicy.InstancePolicy()
    policy.machine_type = "e2-standard-4"
    instances = batch_v1.AllocationPolicy.InstancePolicyOrTemplate()
    instances.policy = policy
    allocation_policy = batch_v1.AllocationPolicy()
    allocation_policy.instances = [instances]

    # Create the job and assign the task group and allocation policy to it
    job = batch_v1.Job()
    job.task_groups = [group]
    job.allocation_policy = allocation_policy

    # Set the labels for the job
    job.labels = labels

    # We use Cloud Logging as it's an out of the box available option
    job.logs_policy = batch_v1.LogsPolicy()
    job.logs_policy.destination = batch_v1.LogsPolicy.Destination.CLOUD_LOGGING

    # Create the job request and set the job and job ID
    create_request = batch_v1.CreateJobRequest()
    create_request.job = job
    create_request.job_id = job_name
    # The job's parent is the region in which the job will run
    create_request.parent = f"projects/{project_id}/locations/{region}"

    return client.create_job(create_request)

Définir des libellés personnalisés pour les exécutables

Les libellés définis dans le labels champ pour un exécutable ne sont appliqués qu'à cet exécutable.

Vous pouvez définir des libellés pour un ou plusieurs exécutables lorsque vous créez un job à l'aide de gcloud CLI ou de l'API Batch.

gcloud

Par exemple, pour créer un job dans us-central1 qui définit deux libellés personnalisés, un pour chacun des deux exécutables du job, procédez comme suit :

  1. Créez un fichier JSON qui spécifie les détails de configuration du job et les runnables.labels champs.

    {
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "container": {
              "imageUri": "gcr.io/google-containers/busybox",
              "entrypoint": "/bin/sh",
              "commands": [
                "-c",
                "echo Hello from task ${BATCH_TASK_INDEX}!"
              ]
            },
            "labels": {
              "RUNNABLE1_LABEL_NAME1": "RUNNABLE1_LABEL_VALUE1"
            }
          },
          {
            "script": {
              "text": "echo Hello from task ${BATCH_TASK_INDEX}!"
            },
            "labels": {
              "RUNNABLE2_LABEL_NAME1": "RUNNABLE2_LABEL_VALUE1"
            }
          }
        ]
      }
    }
  ]
}

    Remplacez les éléments suivants :

    • RUNNABLE1_LABEL_NAME1: nom du libellé à appliquer au premier exécutable du job.

    • RUNNABLE1_LABEL_VALUE1: valeur du libellé à appliquer au premier exécutable du job.

    • RUNNABLE2_LABEL_NAME1: nom du libellé à appliquer au deuxième exécutable du job.

    • RUNNABLE2_LABEL_VALUE1: valeur du libellé à appliquer au deuxième exécutable du job.

  2. Créez le job dans us-central1 à l'aide de la gcloud batch jobs submit commande.

    gcloud batch jobs submit example-job \
    --config=JSON_CONFIGURATION_FILE \
    --location=us-central1

    Remplacez JSON_CONFIGURATION_FILE par le chemin d'accès au fichier JSON contenant les détails de configuration du job que vous avez créé à l'étape précédente.

API

Par exemple, pour créer un job dans us-central1 qui définit deux libellés personnalisés, un pour chacun des deux exécutables du job, envoyez une requête POST à la jobs.create méthode et spécifiez les runnables.labels champs.

POST https://batch.googleapis.com/v1/projects/example-project/locations/us-central1/jobs?job_id=example-job

{
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "container": {
              "imageUri": "gcr.io/google-containers/busybox",
              "entrypoint": "/bin/sh",
              "commands": [
                "-c",
                "echo Hello from ${BATCH_TASK_INDEX}!"
              ]
            },
            "labels": {
              "RUNNABLE1_LABEL_NAME1": "RUNNABLE1_LABEL_VALUE1"
            }
          },
          {
            "script": {
              "text": "echo Hello from ${BATCH_TASK_INDEX}!"
            },
            "labels": {
              "RUNNABLE2_LABEL_NAME1": "RUNNABLE2_LABEL_VALUE1"
            }
          }
        ]
      }
    }
  ]
}

Remplacez les éléments suivants :

  • RUNNABLE1_LABEL_NAME1: nom du libellé à appliquer au premier exécutable du job.

  • RUNNABLE1_LABEL_VALUE1: valeur du libellé à appliquer au premier exécutable du job.

  • RUNNABLE2_LABEL_NAME1: nom du libellé à appliquer au deuxième exécutable du job.

  • RUNNABLE2_LABEL_VALUE1: valeur du libellé à appliquer au deuxième exécutable du job.

Java 


import com.google.cloud.batch.v1.BatchServiceClient;
import com.google.cloud.batch.v1.ComputeResource;
import com.google.cloud.batch.v1.CreateJobRequest;
import com.google.cloud.batch.v1.Job;
import com.google.cloud.batch.v1.LogsPolicy;
import com.google.cloud.batch.v1.Runnable;
import com.google.cloud.batch.v1.TaskGroup;
import com.google.cloud.batch.v1.TaskSpec;
import com.google.protobuf.Duration;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

public class CreateBatchRunnableLabel {
  public static void main(String[] args)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // TODO(developer): Replace these variables before running the sample.
    // Project ID or project number of the Google Cloud project you want to use.
    String projectId = "YOUR_PROJECT_ID";
    // Name of the region you want to use to run the job. Regions that are
    // available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
    String region = "us-central1";
    // The name of the job that will be created.
    // It needs to be unique for each project and region pair.
    String jobName = "example-job";
    // Name of the label1 to be applied for your Job.
    String labelName1 = "RUNNABLE_LABEL_NAME1";
    // Value for the label1 to be applied for your Job.
    String labelValue1 = "RUNNABLE_LABEL_VALUE1";
    // Name of the label2 to be applied for your Job.
    String labelName2 = "RUNNABLE_LABEL_NAME2";
    // Value for the label2 to be applied for your Job.
    String labelValue2 = "RUNNABLE_LABEL_VALUE2";

    createBatchRunnableLabel(projectId, region, jobName, labelName1,
        labelValue1, labelName2, labelValue2);
  }

  // Creates a job with labels defined in the labels field
  // for a runnable. The labels are only applied to that runnable.
  // In Batch, a runnable represents a single task or unit of work within a job.
  // It can be a container (like a Docker image) or a script.
  public static Job createBatchRunnableLabel(String projectId, String region, String jobName,
                   String labelName1, String labelValue1, String labelName2, String labelValue2)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests.
    try (BatchServiceClient batchServiceClient = BatchServiceClient.create()) {

      // Define what will be done as part of the job.
      Runnable runnable =
          Runnable.newBuilder()
              .setContainer(
                  Runnable.Container.newBuilder()
                      .setImageUri("gcr.io/google-containers/busybox")
                      .setEntrypoint("/bin/sh")
                      .addCommands("-c")
                      .addCommands(
                          "echo Hello world! This is task ${BATCH_TASK_INDEX}. "
                              + "This job has a total of ${BATCH_TASK_COUNT} tasks.")
                      .build())
              // Label and its value to be applied to the container
              // that processes data from a specific region.
              .putLabels(labelName1, labelValue1)
              .setScript(Runnable.Script.newBuilder()
              .setText("echo Hello world! This is task ${BATCH_TASK_INDEX}. ").build())
              // Label and its value to be applied to the script
              // that performs some analysis on the processed data.
              .putLabels(labelName2, labelValue2)
              .build();

      // We can specify what resources are requested by each task.
      ComputeResource computeResource =
          ComputeResource.newBuilder()
              // In milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
              .setCpuMilli(2000)
              // In MiB.
              .setMemoryMib(2000)
              .build();

      TaskSpec task =
          TaskSpec.newBuilder()
              // Jobs can be divided into tasks. In this case, we have only one task.
              .addRunnables(runnable)
              .setComputeResource(computeResource)
              .setMaxRetryCount(2)
              .setMaxRunDuration(Duration.newBuilder().setSeconds(3600).build())
              .build();

      // Tasks are grouped inside a job using TaskGroups.
      // Currently, it's possible to have only one task group.
      TaskGroup taskGroup = TaskGroup.newBuilder().setTaskCount(1).setTaskSpec(task).build();

      Job job =
          Job.newBuilder()
              .addTaskGroups(taskGroup)
              // We use Cloud Logging as it's an out of the box available option.
              .setLogsPolicy(LogsPolicy.newBuilder()
              .setDestination(LogsPolicy.Destination.CLOUD_LOGGING).build())
              .build();

      CreateJobRequest createJobRequest =
          CreateJobRequest.newBuilder()
              // The job's parent is the region in which the job will run for the specific project.
              .setParent(String.format("projects/%s/locations/%s", projectId, region))
              .setJob(job)
              .setJobId(jobName)
              .build();

      Job result =
          batchServiceClient
              .createJobCallable()
              .futureCall(createJobRequest)
              .get(5, TimeUnit.MINUTES);

      System.out.printf("Successfully created the job: %s", result.getName());

      return result;
    }
  }

}

Node.js 

// Imports the Batch library
const batchLib = require('@google-cloud/batch');
const batch = batchLib.protos.google.cloud.batch.v1;

// Instantiates a client
const batchClient = new batchLib.v1.BatchServiceClient();

/**
 * TODO(developer): Update these variables before running the sample.
 */
// Project ID or project number of the Google Cloud project you want to use.
const projectId = await batchClient.getProjectId();
// Name of the region you want to use to run the job. Regions that are
// available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
const region = 'us-central1';
// The name of the job that will be created.
// It needs to be unique for each project and region pair.
const jobName = 'example-job';
// Name of the label1 to be applied for your Job.
const labelName1 = 'RUNNABLE_LABEL_NAME1';
// Value for the label1 to be applied for your Job.
const labelValue1 = 'RUNNABLE_LABEL_VALUE1';
// Name of the label2 to be applied for your Job.
const labelName2 = 'RUNNABLE_LABEL_NAME2';
// Value for the label2 to be applied for your Job.
const labelValue2 = 'RUNNABLE_LABEL_VALUE2';

const container = new batch.Runnable.Container({
  imageUri: 'gcr.io/google-containers/busybox',
  entrypoint: '/bin/sh',
  commands: ['-c', 'echo Hello world! This is task ${BATCH_TASK_INDEX}.'],
});

const script = new batch.Runnable.Script({
  commands: ['-c', 'echo Hello world! This is task ${BATCH_TASK_INDEX}.'],
});

const runnable1 = new batch.Runnable({
  container,
  // Label and its value to be applied to the container
  // that processes data from a specific region.
  labels: {
    [labelName1]: labelValue1,
  },
});

const runnable2 = new batch.Runnable({
  script,
  // Label and its value to be applied to the script
  // that performs some analysis on the processed data.
  labels: {
    [labelName2]: labelValue2,
  },
});

// Specify what resources are requested by each task.
const computeResource = new batch.ComputeResource({
  // In milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
  cpuMilli: 500,
  // In MiB.
  memoryMib: 16,
});

const task = new batch.TaskSpec({
  runnables: [runnable1, runnable2],
  computeResource,
  maxRetryCount: 2,
  maxRunDuration: {seconds: 3600},
});

// Tasks are grouped inside a job using TaskGroups.
const group = new batch.TaskGroup({
  taskCount: 3,
  taskSpec: task,
});

const job = new batch.Job({
  name: jobName,
  taskGroups: [group],
  // We use Cloud Logging as it's an option available out of the box
  logsPolicy: new batch.LogsPolicy({
    destination: batch.LogsPolicy.Destination.CLOUD_LOGGING,
  }),
});

// The job's parent is the project and region in which the job will run
const parent = `projects/${projectId}/locations/${region}`;

async function callCreateBatchLabelsRunnable() {
  // Construct request
  const request = {
    parent,
    jobId: jobName,
    job,
  };

  // Run request
  const [response] = await batchClient.createJob(request);
  console.log(JSON.stringify(response));
}

await callCreateBatchLabelsRunnable();

Python 

from google.cloud import batch_v1


def create_job_with_custom_runnables_labels(
    project_id: str,
    region: str,
    job_name: str,
    labels: dict,
) -> batch_v1.Job:
    """
    This method creates a Batch job with custom labels for runnable.
    Args:
        project_id (str): project ID or project number of the Cloud project you want to use.
        region (str): name of the region you want to use to run the job. Regions that are
            available for Batch are listed on: https://cloud.google.com/batch/docs/locations
        job_name (str): the name of the job that will be created.
        labels (dict): a dictionary of key-value pairs that will be used as labels
            E.g., {"label_key1": "label_value2"}
    Returns:
        batch_v1.Job: The created Batch job object containing configuration details.
    """
    client = batch_v1.BatchServiceClient()

    runnable = batch_v1.Runnable()
    runnable.display_name = "Script 1"
    runnable.script = batch_v1.Runnable.Script()
    runnable.script.text = "echo Hello world from Script 1 for task ${BATCH_TASK_INDEX}"
    # Add custom labels to the first runnable
    runnable.labels = labels

    # Create a task specification and assign the runnable and volume to it
    task = batch_v1.TaskSpec()
    task.runnables = [runnable]

    # Specify what resources are requested by each task.
    resources = batch_v1.ComputeResource()
    resources.cpu_milli = 2000  # in milliseconds per cpu-second. This means the task requires 2 whole CPUs.
    resources.memory_mib = 16  # in MiB
    task.compute_resource = resources

    task.max_retry_count = 2
    task.max_run_duration = "3600s"

    # Create a task group and assign the task specification to it
    group = batch_v1.TaskGroup()
    group.task_count = 3
    group.task_spec = task

    # Policies are used to define on what kind of virtual machines the tasks will run on.
    # In this case, we tell the system to use "e2-standard-4" machine type.
    # Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
    policy = batch_v1.AllocationPolicy.InstancePolicy()
    policy.machine_type = "e2-standard-4"
    instances = batch_v1.AllocationPolicy.InstancePolicyOrTemplate()
    instances.policy = policy
    allocation_policy = batch_v1.AllocationPolicy()
    allocation_policy.instances = [instances]

    # Create the job and assign the task group and allocation policy to it
    job = batch_v1.Job()
    job.task_groups = [group]
    job.allocation_policy = allocation_policy

    # We use Cloud Logging as it's an out of the box available option
    job.logs_policy = batch_v1.LogsPolicy()
    job.logs_policy.destination = batch_v1.LogsPolicy.Destination.CLOUD_LOGGING

    # Create the job request and set the job and job ID
    create_request = batch_v1.CreateJobRequest()
    create_request.job = job
    create_request.job_id = job_name
    # The job's parent is the region in which the job will run
    create_request.parent = f"projects/{project_id}/locations/{region}"

    return client.create_job(create_request)

Étape suivante