本教學課程說明如何使用 Terraform,透過 Cloud Scheduler Cron 工作建立及執行 Batch 工作。
Terraform 是一項開放原始碼工具,可讓您在設定檔中指定所需狀態,藉此佈建及管理基礎架構。這些檔案可以視為程式碼,並儲存在 GitHub 等版本管控系統中。
雖然 Terraform 沒有 Batch 資源,但本教學課程會說明如何使用 Terraform 建立 Batch 工作。具體來說,您可以使用 Terraform 安排及執行 Cloud Scheduler Cron 工作,以 Batch API 為目標,建立及執行 Batch 工作。Cloud Scheduler 是一項 Google Cloud 服務,可自動排定 Cron 工作,並支援 Terraform。
本教學課程適用於已使用 Terraform 管理基礎架構,並想將 Batch 作業併入 Terraform 的 Batch 使用者。
建立 Terraform 目錄和設定檔
建立 Terraform 目錄和設定檔,定義要使用 Terraform 建立或更新的資源。本教學課程的範例設定檔定義了名為 batch-job-invoker
的 Cloud Scheduler Cron 工作。啟用後,batch-job-invoker
cron 工作每 5 分鐘就會執行一次,建立定義的 Batch 工作新例項。
如要在該目錄中建立目錄和新的 Terraform 設定 (
.tf
) 檔案,請輸入下列指令,然後按下Enter
:mkdir terraform && cd terraform && cat > main.tf
這個指令會建立
terraform
目錄、將您導向該目錄,並在下一行開始定義新的main.tf
設定檔。複製並貼上下列 Terraform 設定:
# define variables variable "project_id" { type = string description = "The project name to use." default = "PROJECT_ID" } variable "project_number" { type = string description = "The project number to use." default = "PROJECT_NUMBER" } variable "region" { type = string description = "The region where resources are created." default = "us-central1" } variable "cloud_scheduler_service_account_email" { type = string description = "The service account email." default = "CLOUD_SCHEDULER_SERVICE_ACCOUNT_EMAIL" } variable "batch_service_account_email" { type = string description = "The service account email." default = "BATCH_SERVICE_ACCOUNT_EMAIL" } # define a Cloud Scheduler cron job which triggers Batch jobs resource "google_cloud_scheduler_job" "batch-job-invoker" { paused = false # this cron job is enabled name = "batch-job-invoker" project = var.project_id region = var.region schedule = "*/5 * * * *" # when enabled, run every 5 minutes time_zone = "America/Los_Angeles" attempt_deadline = "180s" retry_config { max_doublings = 5 max_retry_duration = "0s" max_backoff_duration = "3600s" min_backoff_duration = "5s" } # when this cron job runs, create and run a Batch job http_target { http_method = "POST" uri = "https://batch.googleapis.com/v1/projects/${var.project_id}/locations/${var.region}/jobs" headers = { "Content-Type" = "application/json" "User-Agent" = "Google-Cloud-Scheduler" } # Batch job definition body = base64encode(<<EOT { "taskGroups":[ { "taskSpec": { "runnables":{ "script": { "text": "echo Hello world! This job was created using Terraform and Cloud Scheduler." } } } } ], "allocationPolicy": { "serviceAccount": { "email": "${var.batch_service_account_email}" } }, "labels": { "source": "terraform_and_cloud_scheduler_tutorial" }, "logsPolicy": { "destination": "CLOUD_LOGGING" } } EOT ) oauth_token { scope = "https://www.googleapis.com/auth/cloud-platform" service_account_email = var.cloud_scheduler_service_account_email } } }
更改下列內容:
PROJECT_ID
:專案的專案 ID。PROJECT_NUMBER
:專案的專案編號。CLOUD_SCHEDULER_SERVICE_ACCOUNT_EMAIL
:您為 Cloud Scheduler Cron 工作準備的服務帳戶電子郵件地址。舉例來說,如要使用 Compute Engine 預設服務帳戶,請指定下列項目:
PROJECT_NUMBER-compute@developer.gserviceaccount.com
BATCH_SERVICE_ACCOUNT_EMAIL
:您為 Batch 工作準備的服務帳戶電子郵件地址。舉例來說,如要使用 Compute Engine 預設服務帳戶,請指定下列項目:
PROJECT_NUMBER-compute@developer.gserviceaccount.com
這項 Terraform 設定定義了一些輸入變數和 cron 工作,用於呼叫 API 方法來建立 Batch 工作。
如要儲存並關閉檔案,請按
Ctrl+D
鍵 (或 macOS 上的Command+D
鍵)。
部署 Terraform 設定,建立 Cron 工作
初始化 Terraform、產生計畫變更,並套用這些變更,即可部署 Terraform 設定。部署 Terraform 設定後,您可以描述專案中的資源,確認 Terraform 是否已順利建立 batch-job-invoker
cron 工作。
在目錄中初始化 Terraform:
terraform init
輸出結果會與下列內容相似:
... Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.
根據專案的目前狀態和設定檔,產生 Terraform 執行計畫:
terraform plan
輸出內容會與以下所示內容類似,顯示系統預計建立
batch-job-invoker
cron 工作:Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # google_cloud_scheduler_job.batch-job-invoker will be created + resource "google_cloud_scheduler_job" "batch-job-invoker" { + id = (known after apply) + name = "batch-job-invoker" + paused = false + project = "PROJECT_ID" + region = "us-central1" + schedule = "*/5 * * * *" + state = (known after apply) + time_zone = "America/Los_Angeles" + http_target { + body = "..." + headers = { + "Content-Type" = "application/json" + "User-Agent" = "Google-Cloud-Scheduler" } + http_method = "POST" + uri = "https://batch.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/jobs" + oauth_token { + scope = "https://www.googleapis.com/auth/cloud-platform" + service_account_email = "CLOUD_SCHEDULER_SERVICE_ACCOUNT_EMAIL" } } + retry_config { + max_backoff_duration = "3600s" + max_doublings = 5 + max_retry_duration = "0s" + min_backoff_duration = "5s" + retry_count = (known after apply) } } Plan: 1 to add, 0 to change, 0 to destroy. ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.
如要套用計畫來建立
batch-job-invoker
cron 工作,請按照下列步驟操作:輸入下列指令:
terraform apply
輸出內容與先前的
terraform plan
指令類似, 但結尾會顯示確認提示。如要確認並套用方案,請輸入
yes
。輸出結果會與下列內容相似:
google_cloud_scheduler_job.batch-job-invoker: Creating... google_cloud_scheduler_job.batch-job-invoker: Creation complete after 0s [id=projects/PROJECT_ID/locations/us-central1/jobs/batch-job-invoker] Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
如要確認
batch-job-invoker
cron 工作是否存在並已啟用,請說明該工作:gcloud scheduler jobs describe batch-job-invoker --location us-central1
輸出結果會與下列內容相似:
attemptDeadline: 180s httpTarget: body: ... headers: Content-Type: application/json User-Agent: Google-Cloud-Scheduler httpMethod: POST oauthToken: scope: https://www.googleapis.com/auth/cloud-platform serviceAccountEmail: CLOUD_SCHEDULER_SERVICE_ACCOUNT_EMAIL uri: https://batch.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/jobs lastAttemptTime: '...' name: projects/PROJECT_ID/locations/us-central1/jobs/batch-job-invoker retryConfig: maxBackoffDuration: 3600s maxDoublings: 5 maxRetryDuration: 0s minBackoffDuration: 5s schedule: '*/5 * * * *' scheduleTime: '...' state: ENABLED status: {} timeZone: America/Los_Angeles userUpdateTime: '...'
在輸出內容中,確認
state
欄位已設為ENABLED
。
確認 Cron 工作是否會建立 Batch 工作
確認 batch-job-invoker
cron 工作是否正確建立批次工作。
您可以等待 5 分鐘,讓系統自動執行 Cron 工作,也可以立即觸發 Cron 工作:
gcloud scheduler jobs run batch-job-invoker --location us-central1
列出由
batch-job-invoker
cron 工作建立的 Batch 工作:gcloud batch jobs list \ --filter labels.source=\"terraform_and_cloud_scheduler_tutorial\" \ --sort-by ~createTime
--filter labels.source=\"terraform_and_cloud_scheduler_tutorial\"
標記會篩選清單,只納入具有標籤 (鍵為source
,值為terraform_and_cloud_scheduler_tutorial
) 的批次工作。--sort-by ~createTime
旗標會將清單從最新到最舊排序。
更新 Terraform 設定,暫停 Cron 工作
在您取得所需數量的 Batch 工作後,請更新並部署 Terraform 設定,以暫停 batch-job-invoker
cron 工作。如要更新 Cron 工作或其他屬性,或日後的批次工作,也適用相同程序。
更新 Terraform 設定檔,將
paused
欄位設為true
,暫停 Cron 工作:sed -i 's/paused = false # this cron job is enabled/paused = true # this cron job is paused/g' main.tf
根據專案的目前狀態和設定檔,產生 Terraform 執行計畫:
terraform plan
輸出內容會與下列內容類似,顯示計畫是將
paused
欄位的值從false
更新為true
:google_cloud_scheduler_job.batch-job-invoker: Refreshing state... [id=projects/PROJECT_ID/locations/us-central1/jobs/batch-job-invoker] Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: ~ update in-place Terraform will perform the following actions: # google_cloud_scheduler_job.batch-job-invoker will be updated in-place ~ resource "google_cloud_scheduler_job" "batch-job-invoker" { id = "projects/PROJECT_ID/locations/us-central1/jobs/batch-job-invoker" name = "batch-job-invoker" ~ paused = false -> true # (6 unchanged attributes hidden) ~ http_target { ~ headers = { + "User-Agent" = "Google-Cloud-Scheduler" # (1 unchanged element hidden) } # (3 unchanged attributes hidden) # (1 unchanged block hidden) } # (1 unchanged block hidden) } Plan: 0 to add, 1 to change, 0 to destroy. ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.
如要套用計畫來更新
batch-job-invoker
cron 工作,請按照下列步驟操作:輸入下列指令:
terraform apply
輸出內容與先前的
terraform plan
指令類似, 但結尾會顯示確認提示。如要確認並套用方案,請輸入
yes
。輸出結果會與下列內容相似:
google_cloud_scheduler_job.batch-job-invoker: Modifying... [id=projects/PROJECT_ID/locations/us-central1/jobs/batch-job-invoker] google_cloud_scheduler_job.batch-job-invoker: Modifications complete after 1s [id=projects/PROJECT_ID/locations/us-central1/jobs/batch-job-invoker] Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
如要確認
batch-job-invoker
cron 工作已暫停,請說明該工作:gcloud scheduler jobs describe batch-job-invoker --location us-central1
輸出結果會與下列內容相似:
attemptDeadline: 180s httpTarget: body: ... headers: Content-Type: application/json User-Agent: Google-Cloud-Scheduler httpMethod: POST oauthToken: scope: https://www.googleapis.com/auth/cloud-platform serviceAccountEmail: CLOUD_SCHEDULER_SERVICE_ACCOUNT_EMAIL uri: https://batch.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/jobs lastAttemptTime: '...' name: projects/PROJECT_ID/locations/us-central1/jobs/batch-job-invoker retryConfig: maxBackoffDuration: 3600s maxDoublings: 5 maxRetryDuration: 0s minBackoffDuration: 5s schedule: '*/5 * * * *' scheduleTime: '...' state: PAUSED status: {} timeZone: America/Los_Angeles userUpdateTime: '...'
在輸出內容中,確認
state
欄位已設為PAUSED
。