Menulis DAG Airflow

Managed Airflow (Gen 3) | Managed Airflow (Gen 2) | Managed Airflow (Legacy Gen 1)

Panduan ini menunjukkan cara menulis directed acyclic graph (DAG) Apache Airflow yang berjalan di lingkungan Managed Service untuk Apache Airflow.

Karena Apache Airflow tidak menyediakan isolasi DAG dan tugas yang kuat, sebaiknya gunakan lingkungan produksi dan pengujian terpisah untuk mencegah gangguan DAG. Untuk mengetahui informasi selengkapnya, lihat Menguji DAG.

Menyusun DAG Airflow

DAG Airflow ditentukan dalam file Python dan terdiri dari komponen berikut:

  • Definisi DAG
  • Operator Airflow
  • Hubungan operator

Cuplikan kode berikut menunjukkan contoh setiap komponen di luar konteks.

Definisi DAG

Contoh berikut menunjukkan definisi DAG Airflow:

import datetime

from airflow import models

default_dag_args = {
    # The start_date describes when a DAG is valid / can be run. Set this to a
    # fixed point in time rather than dynamically, since it is evaluated every
    # time a DAG is parsed. See:
    # https://airflow.apache.org/faq.html#what-s-the-deal-with-start-date
    "start_date": datetime.datetime(2018, 1, 1),
}

# Define a DAG (directed acyclic graph) of tasks.
# Any task you create within the context manager is automatically added to the
# DAG object.
with models.DAG(
    "composer_sample_simple_greeting",
    schedule_interval=datetime.timedelta(days=1),
    default_args=default_dag_args,
) as dag:

Operator dan tugas

Operator Airflow menjelaskan tugas yang akan dilakukan. Tugas task adalah instance spesifik dari operator.

from airflow.operators.bash import BashOperator
from airflow.operators.python import PythonOperator

    def greeting():
        import logging

        logging.info("Hello World!")

    # An instance of an operator is called a task. In this case, the
    # hello_python task calls the "greeting" Python function.
    hello_python = PythonOperator(task_id="hello", python_callable=greeting)

    # Likewise, the goodbye_bash task calls a Bash script.
    goodbye_bash = BashOperator(task_id="bye", bash_command="echo Goodbye.")

Hubungan tugas

Hubungan tugas menjelaskan urutan pengerjaan tugas.

# Define the order in which the tasks complete by using the >> and <<
# operators. In this example, hello_python executes before goodbye_bash.
hello_python >> goodbye_bash

Contoh alur kerja DAG lengkap di Python

Alur kerja berikut adalah template DAG yang berfungsi penuh dan terdiri dari dua tugas: tugas hello_python dan tugas goodbye_bash:


import datetime

from airflow import models

from airflow.operators.bash import BashOperator
from airflow.operators.python import PythonOperator



default_dag_args = {
    # The start_date describes when a DAG is valid / can be run. Set this to a
    # fixed point in time rather than dynamically, since it is evaluated every
    # time a DAG is parsed. See:
    # https://airflow.apache.org/faq.html#what-s-the-deal-with-start-date
    "start_date": datetime.datetime(2018, 1, 1),
}

# Define a DAG (directed acyclic graph) of tasks.
# Any task you create within the context manager is automatically added to the
# DAG object.
with models.DAG(
    "composer_sample_simple_greeting",
    schedule_interval=datetime.timedelta(days=1),
    default_args=default_dag_args,
) as dag:
    def greeting():
        import logging

        logging.info("Hello World!")

    # An instance of an operator is called a task. In this case, the
    # hello_python task calls the "greeting" Python function.
    hello_python = PythonOperator(task_id="hello", python_callable=greeting)

    # Likewise, the goodbye_bash task calls a Bash script.
    goodbye_bash = BashOperator(task_id="bye", bash_command="echo Goodbye.")

    # Define the order in which the tasks complete by using the >> and <<
    # operators. In this example, hello_python executes before goodbye_bash.
    hello_python >> goodbye_bash

Untuk mengetahui informasi selengkapnya tentang cara menentukan DAG Airflow, lihat tutorial Airflow dan konsep Airflow.

Operator Airflow

Contoh berikut menunjukkan beberapa operator Airflow populer. Untuk referensi resmi operator Airflow, lihat Referensi Operator dan Hook dan Indeks penyedia.

BashOperator

Gunakan BashOperator untuk menjalankan program command line.

from airflow.operators import bash

    # Create BigQuery output dataset.
    make_bq_dataset = bash.BashOperator(
        task_id="make_bq_dataset",
        # Executing 'bq' command requires Google Cloud SDK which comes
        # preinstalled in Cloud Composer.
        bash_command=f"bq ls {bq_dataset_name} || bq mk {bq_dataset_name}",
    )

Managed Airflow menjalankan perintah yang diberikan dalam skrip Bash di pekerja Airflow. Worker adalah container Docker berbasis Debian dan mencakup beberapa paket.

PythonOperator

Gunakan PythonOperator untuk menjalankan kode Python arbitrer.

Managed Airflow menjalankan kode Python dalam container yang mencakup paket untuk versi image Managed Airflow yang digunakan di lingkungan Anda.

Untuk menginstal paket Python tambahan, lihat bagian Menginstal Dependensi Python.

Google Cloud Operator

Untuk menjalankan tugas yang menggunakan produk Google Cloud , gunakan operator AirflowGoogle Cloud . Misalnya, operator BigQuery mengkueri dan memproses data di BigQuery.

Ada banyak lagi operator Airflow untuk Google Cloud dan layanan individual yang disediakan oleh Google Cloud. Lihat Google Cloud Operator untuk mengetahui daftar lengkapnya.

from airflow.providers.google.cloud.operators import bigquery
from airflow.providers.google.cloud.transfers import bigquery_to_gcs

    bq_recent_questions_query = bigquery.BigQueryInsertJobOperator(
        task_id="bq_recent_questions_query",
        configuration={
            "query": {
                "query": RECENT_QUESTIONS_QUERY,
                "useLegacySql": False,
                "destinationTable": {
                    "projectId": project_id,
                    "datasetId": bq_dataset_name,
                    "tableId": bq_recent_questions_table_id,
                },
            }
        },
        location=location,
    )

EmailOperator

Gunakan EmailOperator untuk mengirim email dari DAG. Untuk mengirim email dari lingkungan Managed Airflow, konfigurasi lingkungan Anda agar menggunakan SendGrid.

from airflow.operators import email

    # Send email confirmation (you will need to set up the email operator
    # See https://cloud.google.com/composer/docs/how-to/managing/creating#notification
    # for more info on configuring the email operator in Cloud Composer)
    email_summary = email.EmailOperator(
        task_id="email_summary",
        to="{{var.value.email}}",
        subject="Sample BigQuery notify data ready",
        html_content="""
        Analyzed Stack Overflow posts data from {min_date} 12AM to {max_date}
        12AM. The most popular question was '{question_title}' with
        {view_count} views. Top 100 questions asked are now available at:
        {export_location}.
        """.format(
            min_date=min_query_date,
            max_date=max_query_date,
            question_title=(
                "{{ ti.xcom_pull(task_ids='bq_read_most_popular', "
                "key='return_value')[0][0] }}"
            ),
            view_count=(
                "{{ ti.xcom_pull(task_ids='bq_read_most_popular', "
                "key='return_value')[0][1] }}"
            ),
            export_location=output_file,
        ),
    )

Notifikasi saat operator gagal

Tetapkan email_on_failure ke True untuk mengirim notifikasi email saat operator di DAG gagal. Untuk mengirim notifikasi email dari lingkungan Managed Airflow, Anda harus mengonfigurasi lingkungan Anda agar menggunakan SendGrid.

from airflow import models

default_dag_args = {
    "start_date": yesterday,
    # Email whenever an Operator in the DAG fails.
    "email": "{{var.value.email}}",
    "email_on_failure": True,
    "email_on_retry": False,
    "retries": 1,
    "retry_delay": datetime.timedelta(minutes=5),
    "project_id": project_id,
}

with models.DAG(
    "composer_sample_bq_notify",
    schedule_interval=datetime.timedelta(weeks=4),
    default_args=default_dag_args,
) as dag:

Panduan alur kerja DAG

  • Tempatkan library Python kustom di arsip ZIP DAG dalam direktori bertingkat. Jangan menempatkan library di tingkat teratas direktori DAG.

    Saat memindai folder dags/, Airflow hanya memeriksa DAG dalam modul Python yang berada di tingkat teratas folder DAG dan di tingkat teratas arsip ZIP yang juga berada di folder dags/ tingkat teratas. Jika Airflow menemukan modul Python dalam arsip ZIP yang tidak berisi substring airflow dan DAG, Airflow akan berhenti memproses arsip ZIP. Airflow hanya menampilkan DAG yang ditemukan hingga saat itu.

  • Untuk toleransi kesalahan, jangan tentukan beberapa objek DAG dalam modul Python yang sama.

  • Jangan gunakan SubDAG. Sebagai gantinya, kelompokkan tugas di dalam DAG.

  • Tempatkan file yang diperlukan pada waktu penguraian DAG ke dalam folder dags/, bukan di folder data/.

  • Terapkan pengujian unit untuk DAG Anda.

  • Uji DAG yang dikembangkan atau diubah seperti yang direkomendasikan dalam petunjuk untuk menguji DAG.

  • Alat CLI Pengembangan Lokal Composer menyederhanakan pengembangan DAG Apache Airflow untuk Managed Airflow (Gen 2) dengan menjalankan lingkungan Airflow secara lokal. Lingkungan Airflow lokal ini menggunakan image versi Managed Airflow (Gen 2) tertentu.

  • Pastikan DAG yang dikembangkan tidak meningkatkan waktu parsing DAG terlalu banyak.

  • Tugas Airflow dapat gagal karena berbagai alasan. Untuk menghindari kegagalan seluruh proses DAG, sebaiknya aktifkan percobaan ulang tugas. Menyetel percobaan ulang maksimum ke 0 berarti tidak ada percobaan ulang yang dilakukan.

    Sebaiknya ganti opsi default_task_retries dengan nilai untuk coba lagi tugas selain 0. Selain itu, Anda dapat menetapkan parameter retries di tingkat tugas.

  • Jika Anda ingin menggunakan GPU dalam tugas Airflow, buat cluster GKE terpisah berdasarkan node yang menggunakan mesin dengan GPU. Gunakan GKEStartPodOperator untuk menjalankan tugas Anda.

  • Hindari menjalankan tugas yang menggunakan banyak CPU dan memori di node pool cluster tempat komponen Airflow lainnya (penjadwal, pekerja, server web) berjalan. Sebagai gantinya, gunakan KubernetesPodOperator atau GKEStartPodOperator.

  • Saat men-deploy DAG ke lingkungan, upload hanya file yang benar-benar diperlukan untuk menafsirkan dan mengeksekusi DAG ke folder /dags.

  • Batasi jumlah file DAG di folder /dags.

    Airflow terus mengurai DAG di folder /dags. Penguraian adalah proses yang melakukan loop melalui folder DAG dan jumlah file yang perlu dimuat (dengan dependensinya) memengaruhi performa penguraian DAG dan penjadwalan tugas. Jauh lebih efisien menggunakan 100 file dengan masing-masing 100 DAG daripada 10.000 file dengan masing-masing 1 DAG, sehingga pengoptimalan tersebut direkomendasikan. Pengoptimalan ini merupakan keseimbangan antara waktu parsing dan efisiensi penulisan serta pengelolaan DAG.

    Misalnya, untuk men-deploy 10000 file DAG, Anda dapat membuat 100 file ZIP yang masing-masing berisi 100 file DAG.

    Selain petunjuk di atas, jika Anda memiliki lebih dari 10.000 file DAG, maka membuat DAG secara terprogram mungkin merupakan opsi yang baik. Misalnya, Anda dapat menerapkan satu file DAG Python yang menghasilkan sejumlah objek DAG (misalnya, 20, 100 objek DAG).

  • Hindari penggunaan operator Airflow yang tidak digunakan lagi. Sebagai gantinya, gunakan alternatif terbarunya.

  • Hindari penggunaan CloudLoggingHandler untuk menangani logging. Handler logging ini menyebabkan kelebihan beban database lingkungan dan kegagalan DAG berikutnya dengan error "server menutup koneksi secara tidak terduga".

FAQ untuk menulis DAG

Bagaimana cara meminimalkan pengulangan kode jika saya ingin menjalankan tugas yang sama atau serupa di beberapa DAG?

Sebaiknya tentukan library dan wrapper untuk meminimalkan pengulangan kode.

Bagaimana cara menggunakan kembali kode di antara file DAG?

Letakkan fungsi utilitas Anda di library Python lokal dan impor fungsi tersebut. Anda dapat mereferensikan fungsi di DAG mana pun yang berada di folder dags/ dalam bucket lingkungan Anda.

Bagaimana cara meminimalkan risiko munculnya definisi yang berbeda?

Misalnya, Anda memiliki dua tim yang ingin menggabungkan data mentah ke dalam metrik pendapatan. Tim menulis dua tugas yang sedikit berbeda yang mencapai hal yang sama. Tentukan pustaka untuk bekerja dengan data pendapatan sehingga pelaksana DAG harus mengklarifikasi definisi pendapatan yang sedang digabungkan.

Bagaimana cara menetapkan dependensi antar-DAG?

Hal ini bergantung pada cara Anda ingin menentukan dependensi.

Jika memiliki dua DAG (DAG A dan DAG B) dan ingin DAG B dipicu setelah DAG A, Anda dapat menempatkan TriggerDagRunOperator di akhir DAG A.

Jika DAG B hanya bergantung pada artefak yang dihasilkan DAG A, seperti pesan Pub/Sub, maka sensor mungkin lebih cocok.

Jika DAG B terintegrasi erat dengan DAG A, Anda mungkin dapat menggabungkan kedua DAG menjadi satu DAG.

Bagaimana cara meneruskan ID run unik ke DAG dan tugasnya?

Misalnya, Anda ingin meneruskan nama cluster Dataproc dan jalur file.

Anda dapat membuat ID unik acak dengan menampilkan str(uuid.uuid4()) dalam PythonOperator. Tindakan ini menempatkan ID ke dalam XComs sehingga Anda dapat merujuk ke ID di operator lain melalui kolom yang dibuat dengan template.

Sebelum membuat uuid, pertimbangkan apakah ID khusus DagRun akan lebih berguna. Anda juga dapat mereferensikan ID ini dalam penggantian Jinja dengan menggunakan makro.

Bagaimana cara memisahkan tugas dalam DAG?

Setiap tugas harus berupa unit kerja idempoten. Oleh karena itu, Anda harus menghindari mengenkapsulasi alur kerja multi-langkah dalam satu tugas, seperti program kompleks yang berjalan di PythonOperator.

Haruskah saya menentukan beberapa tugas dalam satu DAG untuk menggabungkan data dari beberapa sumber?

Misalnya, Anda memiliki beberapa tabel dengan data mentah dan ingin membuat agregat harian untuk setiap tabel. Tugas-tugas tersebut tidak saling bergantung. Haruskah Anda membuat satu tugas dan DAG untuk setiap tabel atau membuat satu DAG umum?

Jika Anda tidak masalah dengan setiap tugas yang berbagi properti tingkat DAG yang sama, seperti schedule, maka sebaiknya tentukan beberapa tugas dalam satu DAG. Atau, untuk meminimalkan pengulangan kode, beberapa DAG dapat dibuat dari satu modul Python dengan menempatkannya ke dalam globals() modul.

Bagaimana cara membatasi jumlah tugas serentak yang berjalan dalam DAG?

Misalnya, Anda ingin menghindari melebihi batas/kuota penggunaan API atau menghindari menjalankan terlalu banyak proses secara bersamaan.

Anda dapat menentukan kumpulan Airflow di UI web Airflow dan mengaitkan tugas dengan kumpulan yang ada di DAG.

FAQ terkait penggunaan operator

Haruskah saya menggunakan DockerOperator?

Sebaiknya jangan gunakan DockerOperator, kecuali jika digunakan untuk meluncurkan penampung pada penginstalan Docker jarak jauh (bukan dalam cluster lingkungan). Di lingkungan Managed Airflow, operator tidak memiliki akses ke daemon Docker.

Sebagai gantinya, gunakan KubernetesPodOperator atau GKEStartPodOperator. Operator ini meluncurkan pod Kubernetes ke cluster Kubernetes atau GKE. Perhatikan bahwa kami tidak merekomendasikan peluncuran pod ke cluster lingkungan, karena hal ini dapat menyebabkan persaingan resource.

Haruskah saya menggunakan SubDagOperator?

Sebaiknya jangan gunakan SubDagOperator.

Gunakan alternatif seperti yang disarankan dalam Mengelompokkan tugas.

Haruskah saya menjalankan kode Python hanya di PythonOperators untuk memisahkan sepenuhnya operator Python?

Bergantung pada sasaran Anda, ada beberapa opsi yang tersedia.

Jika satu-satunya kekhawatiran Anda adalah mempertahankan dependensi Python terpisah, Anda dapat menggunakan PythonVirtualenvOperator.

Pertimbangkan penggunaan KubernetesPodOperator. Operator ini memungkinkan Anda menentukan pod Kubernetes dan menjalankan pod di cluster lain.

Bagaimana cara menambahkan paket biner kustom atau non-PyPI?

Anda dapat menginstal paket yang dihosting di repositori paket pribadi.

Bagaimana cara meneruskan argumen secara seragam ke DAG dan tugasnya?

Anda dapat menggunakan dukungan bawaan Airflow untuk template Jinja guna meneruskan argumen yang dapat digunakan di kolom ber-template.

Kapan penggantian template terjadi?

Penggantian template terjadi pada pekerja Airflow tepat sebelum fungsi pre_execute operator dipanggil. Dalam praktiknya, artinya template tidak diganti hingga tepat sebelum tugas berjalan.

Bagaimana cara mengetahui argumen operator mana yang mendukung penggantian template?

Argumen operator yang mendukung penggantian template Jinja2 ditandai secara eksplisit.

Cari kolom template_fields dalam definisi Operator, yang berisi daftar nama argumen yang mengalami penggantian template.

Misalnya, lihat BashOperator, yang mendukung pembuatan template untuk argumen bash_command dan env.

Operator Airflow yang tidak digunakan lagi dan dihapus

Operator Airflow yang tercantum dalam tabel berikut tidak digunakan lagi:

  • Hindari penggunaan operator ini di DAG Anda. Sebagai gantinya, gunakan operator pengganti yang disediakan dan terbaru.

  • Jika operator tercantum sebagai dihapus, operator tersebut sudah tidak tersedia di salah satu versi Managed Airflow (Gen 2) yang dirilis.

  • Jika operator tercantum sebagai tidak digunakan lagi, operator tersebut tidak digunakan lagi dan akan dihapus dalam Managed Airflow (Gen 2) versi mendatang.

Operator yang tidak digunakan lagi Operator penggantian

GCSObjectsWtihPrefixExistenceSensor

Dihapus di 7.0.0.

Versi: composer-3-*, composer-2.0.28-airflow-2.3.3.

GCSObjectsWithPrefixExistenceSensor

Tersedia di 2.0.0.

Versi: composer-3-*, composer-2-*.

CreateAutoMLTextTrainingJobOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

SupervisedFineTuningTrainOperator

Tersedia di 10.23.0.

Versi: composer-3-airflow-2.9.3-build.1, composer-3-airflow-2.9.1-build.8, composer-2.9.5-airflow-2.9.3, composer-2.9.5-airflow-2.9.1.

GKEDeploymentHook

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

GKEKubernetesHook

Tersedia di 10.17.0.

Versi: composer-3-*, composer-2.7.1-airflow-2.7.3.

GKECustomResourceHook

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

GKEKubernetesHook

Tersedia di 10.17.0.

Versi: composer-3-*, composer-2.7.1-airflow-2.7.3.

GKEPodHook

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

GKEKubernetesHook

Tersedia di 10.17.0.

Versi: composer-3-*, composer-2.7.1-airflow-2.7.3.

GKEJobHook

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

GKEKubernetesHook

Tersedia di 10.17.0.

Versi: composer-3-*, composer-2.7.1-airflow-2.7.3.

GKEPodAsyncHook

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

GKEKubernetesAsyncHook

Tersedia di 10.17.0.

Versi: composer-3-*, composer-2.7.1-airflow-2.7.3.

SecretsManagerHook

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

GoogleCloudSecretManagerHook

Tersedia di 10.18.0.

Versi: composer-3-airflow-2.7.3-build.6, composer-2.8.3-airflow-2.7.3.

BigQueryExecuteQueryOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

BigQueryInsertJobOperator

Tersedia di 1.0.0.

Versi: composer-3-*, composer-2-*.

BigQueryPatchDatasetOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

BigQueryUpdateDatasetOperator

Tersedia di 1.0.0.

Versi: composer-3-*, composer-2-*.

DataflowCreateJavaJobOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

beam.BeamRunJavaPipelineOperator

Tersedia di apache-airflow-providers-apache-beam==1.0.0.

Versi: composer-3-*, composer-2-*.

DataflowCreatePythonJobOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

beam.BeamRunPythonPipelineOperator

Tersedia di apache-airflow-providers-apache-beam==1.0.0.

Versi: composer-3-*, composer-2-*.

DataprocSubmitPigJobOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

DataprocSubmitJobOperator

Tersedia di 1.0.0.

Versi: composer-3-*, composer-2-*.

DataprocSubmitHiveJobOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

DataprocSubmitJobOperator

Tersedia di 1.0.0.

Versi: composer-3-*, composer-2-*.

DataprocSubmitSparkSqlJobOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

DataprocSubmitJobOperator

Tersedia di 1.0.0.

Versi: composer-3-*, composer-2-*.

DataprocSubmitSparkJobOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

DataprocSubmitJobOperator

Tersedia di 1.0.0.

Versi: composer-3-*, composer-2-*.

DataprocSubmitHadoopJobOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

DataprocSubmitJobOperator

Tersedia di 1.0.0.

Versi: composer-3-*, composer-2-*.

DataprocSubmitPySparkJobOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

DataprocSubmitJobOperator

Tersedia di 1.0.0.

Versi: composer-3-*, composer-2-*.

BigQueryTableExistenceAsyncSensor

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

BigQueryTableExistenceSensor

Tersedia di 8.12.0.

Versi: composer-3-*, composer-2.3.0-airflow-2.5.1, composer-2.3.0-airflow-2.4.3.

BigQueryTableExistencePartitionAsyncSensor

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

BigQueryTablePartitionExistenceSensor

Tersedia di 8.12.0.

Versi: composer-3-*, composer-2.3.0-airflow-2.5.1, composer-2.3.0-airflow-2.4.3.

CloudComposerEnvironmentSensor

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

CloudComposerCreateEnvironmentOperator, CloudComposerDeleteEnvironmentOperator, CloudComposerUpdateEnvironmentOperator

Tersedia di 8.13.0.

Versi: composer-3-*, composer-2.3.0-airflow-2.5.1, composer-2.3.0-airflow-2.4.3.

GCSObjectExistenceAsyncSensor

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

GCSObjectExistenceSensor

Tersedia di 8.12.0.

Versi: composer-3-*, composer-2.3.0-airflow-2.5.1, composer-2.3.0-airflow-2.4.3.

GoogleAnalyticsHook

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

GoogleAnalyticsAdminHook

Tersedia di 8.14.0.

Versi: composer-3-*, composer-2.3.0-airflow-2.5.1, composer-2.3.0-airflow-2.4.3.

GoogleAnalyticsListAccountsOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

GoogleAnalyticsAdminListAccountsOperator

Tersedia di 8.14.0.

Versi: composer-3-*, composer-2.3.0-airflow-2.5.1, composer-2.3.0-airflow-2.4.3.

GoogleAnalyticsGetAdsLinkOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

GoogleAnalyticsAdminGetGoogleAdsLinkOperator

Tersedia di 8.14.0.

Versi: composer-3-*, composer-2.3.0-airflow-2.5.1, composer-2.3.0-airflow-2.4.3.

GoogleAnalyticsRetrieveAdsLinksListOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

GoogleAnalyticsAdminListGoogleAdsLinksOperator

Tersedia di 8.14.0.

Versi: composer-3-*, composer-2.3.0-airflow-2.5.1, composer-2.3.0-airflow-2.4.3.

GoogleAnalyticsDataImportUploadOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

GoogleAnalyticsAdminCreateDataStreamOperator

Tersedia di 8.14.0.

Versi: composer-3-*, composer-2.3.0-airflow-2.5.1, composer-2.3.0-airflow-2.4.3.

GoogleAnalyticsDeletePreviousDataUploadsOperator

Dihapus di 11.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

GoogleAnalyticsAdminDeleteDataStreamOperator

Tersedia di 8.14.0.

Versi: composer-3-*, composer-2.3.0-airflow-2.5.1, composer-2.3.0-airflow-2.4.3.

DataPipelineHook

Dihapus di 12.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

DataflowHook

Tersedia di 10.20.0.

Versi: composer-3-airflow-2.9.1-build.0, composer-3-airflow-2.7.3-build.9, composer-2.8.6-airflow-2.9.1, composer-2.8.6-airflow-2.7.3.

CreateDataPipelineOperator

Dihapus di 12.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

DataflowCreatePipelineOperator

Tersedia di 10.20.0.

Versi: composer-3-airflow-2.9.1-build.0, composer-3-airflow-2.7.3-build.9, composer-2.8.6-airflow-2.9.1, composer-2.8.6-airflow-2.7.3.

RunDataPipelineOperator

Dihapus di 12.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

DataflowRunPipelineOperator

Tersedia di 10.20.0.

Versi: composer-3-airflow-2.9.1-build.0, composer-3-airflow-2.7.3-build.9, composer-2.8.6-airflow-2.9.1, composer-2.8.6-airflow-2.7.3.

AutoMLBatchPredictOperator

Dihapus di 13.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

vertex_ai.batch_prediction_job

Tersedia di 10.24.0.

Versi: composer-3-airflow-2.9.3-build.4, composer-2.9.8-airflow-2.9.3.

PromptLanguageModelOperator

Dihapus di 13.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

TextGenerationModelPredictOperator

Tersedia di 10.21.0.

Versi: composer-3-airflow-2.9.1-build.0, composer-3-airflow-2.7.3-build.9, composer-2.8.6-airflow-2.9.1, composer-2.8.6-airflow-2.7.3.

GenerateTextEmbeddingsOperator

Dihapus di 13.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

TextEmbeddingModelGetEmbeddingsOperator

Tersedia di 10.21.0.

Versi: composer-3-airflow-2.9.1-build.0, composer-3-airflow-2.7.3-build.9, composer-2.8.6-airflow-2.9.1, composer-2.8.6-airflow-2.7.3.

PromptMultimodalModelOperator

Dihapus di 13.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

GenerativeModelGenerateContentOperator

Tersedia di 10.21.0.

Versi: composer-3-airflow-2.9.1-build.0, composer-3-airflow-2.7.3-build.9, composer-2.8.6-airflow-2.9.1, composer-2.8.6-airflow-2.7.3.

PromptMultimodalModelWithMediaOperator

Dihapus di 13.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

GenerativeModelGenerateContentOperator

Tersedia di 10.21.0.

Versi: composer-3-airflow-2.9.1-build.0, composer-3-airflow-2.7.3-build.9, composer-2.8.6-airflow-2.9.1, composer-2.8.6-airflow-2.7.3.

DataflowStartSqlJobOperator

Dihapus di 13.0.0.

Versi: composer-3-airflow-2.10.5-build.0, composer-3-airflow-2.10.2-build.12, composer-3-airflow-2.9.3-build.19, composer-2.12.1-airflow-2.10.5, composer-2.12.0-airflow-2.10.2, composer-2.12.0-airflow-2.9.3.

DataflowStartYamlJobOperator

Tersedia di 10.23.0.

Versi: composer-3-airflow-2.9.3-build.1, composer-3-airflow-2.9.1-build.8, composer-2.9.5-airflow-2.9.3, composer-2.9.5-airflow-2.9.1.

DataprocScaleClusterOperator

Dihapus di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

DataprocUpdateClusterOperator

Tersedia di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

MLEngineStartBatchPredictionJobOperator

Dihapus di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

CreateBatchPredictionJobOperator

Tersedia di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

MLEngineManageModelOperator

Dihapus di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

MLEngineCreateModelOperator, MLEngineGetModelOperator

Tersedia di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

MLEngineGetModelOperator

Dihapus di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

GetModelOperator

Tersedia di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

MLEngineDeleteModelOperator

Dihapus di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

DeleteModelOperator

Tersedia di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

MLEngineManageVersionOperator

Dihapus di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

MLEngineCreateVersion, MLEngineSetDefaultVersion, MLEngineListVersions, MLEngineDeleteVersion

Tersedia di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

MLEngineCreateVersionOperator

Dihapus di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

Parameter parent_model untuk operator VertexAI

Tersedia di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

MLEngineSetDefaultVersionOperator

Dihapus di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

SetDefaultVersionOnModelOperator

Tersedia di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

MLEngineListVersionsOperator

Dihapus di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

ListModelVersionsOperator

Tersedia di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

MLEngineDeleteVersionOperator

Dihapus di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

DeleteModelVersionOperator

Tersedia di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

MLEngineStartTrainingJobOperator

Dihapus di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

CreateCustomPythonPackageTrainingJobOperator

Tersedia di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

MLEngineTrainingCancelJobOperator

Dihapus di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

CancelCustomTrainingJobOperator

Tersedia di 15.0.0.

Versi: composer-3-airflow-2.9.3-build.23, composer-3-airflow-2.10.5-build.2, composer-2.13.0-airflow-2.10.5, composer-2.13.1-airflow-2.9.3.

AutoMLDatasetLink

Dihapus di 16.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

TranslationLegacyDatasetLink

Tersedia di 10.19.0.

Versi: composer-3-airflow-2.9.1-build.0, composer-3-airflow-2.7.3-build.9, composer-2.8.6-airflow-2.9.1, composer-2.8.6-airflow-2.7.3.

AutoMLDatasetListLink

Dihapus di 16.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

TranslationDatasetListLink

Tersedia di 10.19.0.

Versi: composer-3-airflow-2.9.1-build.0, composer-3-airflow-2.7.3-build.9, composer-2.8.6-airflow-2.9.1, composer-2.8.6-airflow-2.7.3.

AutoMLModelLink

Dihapus di 16.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

TranslationLegacyModelLink

Tersedia di 10.19.0.

Versi: composer-3-airflow-2.9.1-build.0, composer-3-airflow-2.7.3-build.9, composer-2.8.6-airflow-2.9.1, composer-2.8.6-airflow-2.7.3.

AutoMLModelTrainLink

Dihapus di 16.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

TranslationLegacyModelTrainLink

Tersedia di 10.19.0.

Versi: composer-3-airflow-2.9.1-build.0, composer-3-airflow-2.7.3-build.9, composer-2.8.6-airflow-2.9.1, composer-2.8.6-airflow-2.7.3.

AutoMLModelPredictLink

Dihapus di 16.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

TranslationLegacyModelPredictLink

Tersedia di 10.19.0.

Versi: composer-3-airflow-2.9.1-build.0, composer-3-airflow-2.7.3-build.9, composer-2.8.6-airflow-2.9.1, composer-2.8.6-airflow-2.7.3.

TextGenerationModelPredictOperator

Dihapus di 16.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

GenerativeModelGenerateContentOperator

Tersedia di 16.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

LifeSciencesHook

Dihapus di 17.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

Hook Operator Batch Google Cloud

Tersedia di 17.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

LifeSciencesLink

Dihapus di 17.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

Tidak ada pengganti (API dihentikan)

LifeSciencesRunPipelineOperator

Dihapus di 17.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

Operator Batch Google Cloud

Tersedia di 17.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

BigQueryCreateEmptyTableOperator

Dihapus di 17.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

BigQueryCreateTableOperator

Tersedia di 17.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

BigQueryCreateExternalTableOperator

Dihapus di 17.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

BigQueryCreateTableOperator

Tersedia di 17.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

GoogleDisplayVideo360CreateQueryOperator

Dihapus di 18.0.0.

Versi: composer-3-airflow-3.1.0-build.2, composer-3-airflow-2.10.5-build.17, composer-2.15.0-airflow-2.10.5.

display_video.GoogleDisplayVideo360CreateSDFDownloadTaskOperator

Tersedia di 18.0.0.

Versi: composer-3-airflow-3.1.0-build.2, composer-3-airflow-2.10.5-build.17, composer-2.15.0-airflow-2.10.5.

GoogleDisplayVideo360RunQueryOperator

Dihapus di 18.0.0.

Versi: composer-3-airflow-3.1.0-build.2, composer-3-airflow-2.10.5-build.17, composer-2.15.0-airflow-2.10.5.

display_video.GoogleDisplayVideo360CreateSDFDownloadTaskOperator

Tersedia di 18.0.0.

Versi: composer-3-airflow-3.1.0-build.2, composer-3-airflow-2.10.5-build.17, composer-2.15.0-airflow-2.10.5.

GoogleDisplayVideo360DeleteReportOperator

Dihapus di 18.0.0.

Versi: composer-3-airflow-3.1.0-build.2, composer-3-airflow-2.10.5-build.17, composer-2.15.0-airflow-2.10.5.

Tugas ekspor SDF di API v4

Tersedia di 18.0.0.

Versi: composer-3-airflow-3.1.0-build.2, composer-3-airflow-2.10.5-build.17, composer-2.15.0-airflow-2.10.5.

GoogleDisplayVideo360DownloadReportV2Operator

Dihapus di 18.0.0.

Versi: composer-3-airflow-3.1.0-build.2, composer-3-airflow-2.10.5-build.17, composer-2.15.0-airflow-2.10.5.

display_video.GoogleDisplayVideo360SDFtoGCSOperator

Tersedia di 18.0.0.

Versi: composer-3-airflow-3.1.0-build.2, composer-3-airflow-2.10.5-build.17, composer-2.15.0-airflow-2.10.5.

GoogleDisplayVideo360DownloadLineItemsOperator

Dihapus di 18.0.0.

Versi: composer-3-airflow-3.1.0-build.2, composer-3-airflow-2.10.5-build.17, composer-2.15.0-airflow-2.10.5.

display_video.GoogleDisplayVideo360CreateSDFDownloadTaskOperator

Tersedia di 18.0.0.

Versi: composer-3-airflow-3.1.0-build.2, composer-3-airflow-2.10.5-build.17, composer-2.15.0-airflow-2.10.5.

GoogleDisplayVideo360UploadLineItemsOperator

Dihapus di 18.0.0.

Versi: composer-3-airflow-3.1.0-build.2, composer-3-airflow-2.10.5-build.17, composer-2.15.0-airflow-2.10.5.

display_video.GoogleDisplayVideo360SDFtoGCSOperator

Tersedia di 18.0.0.

Versi: composer-3-airflow-3.1.0-build.2, composer-3-airflow-2.10.5-build.17, composer-2.15.0-airflow-2.10.5.

GoogleDisplayVideo360RunQuerySensor

Dihapus di 18.0.0.

Versi: composer-3-airflow-3.1.0-build.2, composer-3-airflow-2.10.5-build.17, composer-2.15.0-airflow-2.10.5.

Tugas ekspor SDF di API v4

Tersedia di 18.0.0.

Versi: composer-3-airflow-3.1.0-build.2, composer-3-airflow-2.10.5-build.17, composer-2.15.0-airflow-2.10.5.

AutoMLPredictOperator

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

TextGenerationModelPredictOperator, TranslateTextOperator

Tersedia di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

CloudAutoMLHook

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

vertex_ai.auto_ml.AutoMLHook, translate.TranslateHook

Tersedia di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

AutoMLCreateDatasetOperator

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

vertex_ai.dataset.CreateDatasetOperator, translate.TranslateCreateDatasetOperator

Tersedia di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

AutoMLDeleteDatasetOperator

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

vertex_ai.dataset.ListDatasetsOperator, translate.TranslateDatasetsListOperator

Tersedia di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

AutoMLDeleteModelOperator

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

vertex_ai.model_service.DeleteModelOperator, translate.TranslateDeleteModelOperator

Tersedia di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

AutoMLDeployModelOperator

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

vertex_ai.endpoint_service.DeployModelOperator

Tersedia di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

AutoMLGetModelOperator

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

vertex_ai.model_service.GetModelOperator

Tersedia di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

AutoMLImportDataOperator

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

vertex_ai.dataset.ImportDataOperator, translate.TranslateImportDataOperator

Tersedia di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

AutoMLListDatasetOperator

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

vertex_ai.dataset.ListDatasetsOperator, translate.TranslateDatasetsListOperator

Tersedia di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

AutoMLTablesListColumnSpecsOperator

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

Tidak ada pengganti (versi lama AutoML Tables dihentikan)

AutoMLTablesListTableSpecsOperator

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

Tidak ada pengganti (versi lama AutoML Tables dihentikan)

AutoMLTablesUpdateDatasetOperator

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

vertex_ai.dataset.UpdateDatasetOperator

Tersedia di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

AutoMLTrainModelOperator

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

vertex_ai.auto_ml.CreateAutoMLTabularTrainingJobOperator, CreateAutoMLVideoTrainingJobOperator, CreateAutoMLImageTrainingJobOperator, SupervisedFineTuningTrainOperator, TranslateCreateModelOperator

Tersedia di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

MLEngineCreateModelOperator

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

operator VertexAI yang sesuai

Tersedia di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

DataCatalogEntryGroupLink

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogEntryGroupLink

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

DataCatalogEntryLink

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogEntryLink

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

DataCatalogTagTemplateLink

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogAspectTypeLink

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogCreateEntryOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogCreateEntryOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogCreateEntryGroupOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogCreateEntryGroupOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogCreateTagOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogCreateEntryOperator, DataplexCatalogUpdateEntryOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogCreateTagTemplateOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogCreateAspectTypeOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogCreateTagTemplateFieldOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogUpdateAspectTypeOperator, DataplexCatalogCreateAspectTypeOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogDeleteEntryGroupOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogDeleteEntryGroupOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogDeleteTagOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogUpdateEntryOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogDeleteTagTemplateOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogDeleteAspectTypeOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogDeleteTagTemplateFieldOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogUpdateAspectTypeOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogGetEntryOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogGetEntryOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogGetEntryGroupOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogGetEntryGroupOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogGetTagTemplateOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogGetAspectTypeOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogListTagsOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogGetEntryOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogLookupEntryOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogLookupEntryOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogRenameTagTemplateFieldOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogUpdateAspectTypeOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogSearchCatalogOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogSearchEntriesOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogUpdateEntryOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogUpdateEntryOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogUpdateTagOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogUpdateEntryOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogUpdateTagTemplateOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogUpdateAspectTypeOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogUpdateTagTemplateFieldOperator

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexCatalogUpdateAspectTypeOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CloudDataCatalogHook

Tidak digunakan lagi di 16.1.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

DataplexHook

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

TextEmbeddingModelGetEmbeddingsOperator

Tidak digunakan lagi di 18.1.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

GenAIGenerateEmbeddingsOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

GenerativeModelGenerateContentOperator

Tidak digunakan lagi di 18.1.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

GenAIGenerateContentOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

SupervisedFineTuningTrainOperator

Tidak digunakan lagi di 18.1.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

GenAISupervisedFineTuningTrainOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CountTokensOperator

Tidak digunakan lagi di 18.1.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

GenAICountTokensOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

CreateCachedContentOperator

Tidak digunakan lagi di 18.1.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

GenAICreateCachedContentOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

GenerateFromCachedContentOperator

Tidak digunakan lagi di 18.1.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

GenAIGenerateContentOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

DeleteExperimentRunOperator

Tidak digunakan lagi di 18.1.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

DeleteExperimentRunOperator

Tersedia di 21.0.0.

Versi: composer-3-airflow-3.1.7-build.11, composer-3-airflow-2.11.1-build.7, composer-2.17.4-airflow-2.11.1.

TranslationLegacyModelLink

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

TranslationModelLink

Tersedia di 16.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

TranslationDatasetListLink

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

TranslationDatasetsListLink

Tersedia di 16.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

TranslationLegacyDatasetLink

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

TranslationDatasetsListLink

Tersedia di 16.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

TranslationLegacyModelTrainLink

Dihapus di 19.0.0.

Versi: composer-3-airflow-2.10.5-build.22, composer-2.16.0-airflow-2.10.5.

TranslationModelsListLink

Tersedia di 16.0.0.

Versi: composer-3-airflow-2.10.5-build.12, composer-2.14.0-airflow-2.10.5.

Langkah berikutnya