Managed Airflow (Gen 3) | Managed Airflow (Gen 2) | Managed Airflow (Legacy Gen 1)
Managed Airflow offers a handful of security features and compliances that are beneficial for enterprise companies with stricter security requirements.
These three sections present information about Managed Airflow security features:
- Basic security features. Describes features that are available in Managed Airflow environments by default.
- Advanced security features. Describes features which you can use to modify Managed Airflow to your security requirements.
- Compliance to standards. Provides a list of standards that Managed Airflow is compliant with.
Basic security features
This section lists security-related features provided by default for each Managed Airflow environment.
Encryption at rest
Managed Airflow utilizes encryption at rest in Google Cloud.
Managed Airflow stores data in different services. For example, the Airflow Metadata DB uses Cloud SQL database, DAGs are stored in Cloud Storage buckets.
By default, data is encrypted using Google-owned and Google-managed encryption keys.
If you prefer, you can configure Managed Airflow environments to be encrypted with customer-managed encryption keys.
Uniform bucket-level access
Uniform bucket-level access allows you to uniformly control access to your Cloud Storage resources. This mechanism also applies to your environment's bucket, which stores your DAGs and plugins.
User permissions
Managed Airflow has several features for managing user permissions:
IAM roles and permissions. Managed Airflow environments in a Google Cloud project can be accessed only by users whose accounts are added to IAM of the project.
Managed Airflow-specific roles and permissions. You assign these roles and permissions to user accounts in your project. Each role defines the types of operations that a user account can perform on Managed Airflow environments in your project.
Airflow UI Access Control. Users in your project can have different access levels in the Airflow UI. This mechanism is called Airflow UI Access Control (Airflow Role-Based Access Control, or Airflow RBAC).
Domain Restricted Sharing (DRS). Managed Airflow supports Domain Restricted Sharing organizational policy. If you use this policy, then only users from the selected domains can access your environments.
Private IP environments
You can create Managed Airflow environments in the Private IP networking configuration.
In the Private IP mode, nodes of your environment's cluster do not have external IP addresses and do not communicate through the public internet.
Your environment's cluster uses Shielded VMs
Shielded VMs are virtual machines (VMs) on Google Cloud hardened by a set of security controls that help defend against rootkits and bootkits.
Managed Airflow environments use Shielded VMs to run the nodes of their environment cluster.
Advanced security features
This section lists advanced security-related features for Managed Airflow environments.
Customer Managed Encryption Keys (CMEK)
Managed Airflow supports Customer Managed Encryption Keys (CMEK). CMEK provide you with more control over the keys used to encrypt data at rest within a Google Cloud project.
You can use CMEK with Managed Airflow to encrypt and decrypt data generated by a Managed Airflow environment.
VPC Service Controls (VPC SC) Support
VPC Service Controls is a mechanism to mitigate data exfiltration risks.
Managed Airflow can be selected as a secured service inside a VPC Service Controls perimeter. All underlying resources used by Managed Airflow are configured to support VPC Service Controls architecture and follow its rules. Only Private IP environments can be created in a VPC SC perimeter.
Deploying Managed Airflow environments with VPC Service Controls gives you:
Reduced risk of data exfiltration.
Protection against data exposure due to misconfigured access controls.
Reduced risk of malicious users copying data to unauthorized Google Cloud resources, or external attackers accessing Google Cloud resources from the internet.
Web server network access control levels (ACL)
Airflow web servers in Managed Airflow are always provisioned with an externally accessible IP address. You can control from which IP addresses the Airflow UI can be accessed. Managed Airflow supports IPv4 and IPv6 ranges.
You can configure web server access restrictions
in Google Cloud console, gcloud, API, and Terraform.
Secret Manager as a storage for sensitive configuration data
In Managed Airflow, you can configure Airflow to use Secret Manager as a backend where Airflow connection variables are stored.
DAG developers can also read variables and connection stored in Secret Manager from the DAG code.
Compliance to standards
See the pages linked below to check Managed Airflow's compliance with various standards:
- HIPAA Compliance
- Access Transparency
- PCI DSS
- ISO/IEC: 27001, 27017, 27018
- SOC: SOC 1, SOC 2, SOC 3
- NIST: NIST800-53, NIST800-171
- DRZ FedRamp Moderate
- Data Residency/Location Restrictions (configuration guide for Managed Airflow)
- Assured Workloads
See also
Some of the security features mentioned in this article are discussed in the the Airflow Summit 2020 presentation: Run Airflow DAGs in a secure way.