In diesem Dokument werden die Funktionen, Konfigurationen und APIs in Managed Service for Apache Spark beschrieben, die mit den Steuerelementen für unterstützte Kontrollpakete übereinstimmen. In diesem Dokument wird davon ausgegangen, dass Sie Assured Workloads verwenden.

Alle einblenden Datengrenze für ITAR

Datengrenze für ITAR Unterstützte Dienste In der folgenden Tabelle sind die APIs und Versionen von Managed Service for Apache Spark aufgeführt, die die Anforderungen der ITAR-Datengrenze erfüllen. Dienst Version Status dataproc.googleapis.com v1 UNTERSTÜTZT dataproc.googleapis.com v1beta2 UNTERSTÜTZT Unterstützte Regionen für Compliance Managed Service for Apache Spark ist für Datengrenze für ITAR in den folgenden Google Cloud Regionen verfügbar: us-central1 us-central2 us-east1 us-east4 us-east5 us-south1 us-west1 us-west2 us-west3 us-west4 API-Felder für vertrauliche Daten Ressource: dataproc.googleapis.com/Batch In der folgenden Tabelle sind die API-Ressourcen und -Felder aufgeführt, die für die Verarbeitung von Daten konzipiert sind, die gemäß ITAR durch die Datengrenze geschützt sind. API-Methode Geschützte Felder Dienst: dataproc.googleapis.com REST API: POST /v1/{parent=projects/*/locations/*}/batches RPC-Methoden: google.cloud.dataproc.v1.BatchController.CreateBatch batch.pysparkBatch.args batch.rayBatch.args batch.sparkBatch.args batch.sparkRBatch.args batch.sparkSqlBatch.queryVariables.key batch.sparkSqlBatch.queryVariables.value Ressource: dataproc.googleapis.com/Job In der folgenden Tabelle sind die API-Ressourcen und -Felder aufgeführt, die für die Verarbeitung von Daten konzipiert sind, die gemäß ITAR durch die Datengrenze geschützt sind. API-Methode Geschützte Felder Dienst: dataproc.googleapis.com REST API: PATCH /v1/projects/{project_id}/regions/{region}/jobs/{job_id} RPC-Methoden: google.cloud.dataproc.v1.JobController.UpdateJob job.flinkJob.args job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pyflinkJob.args job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value job.trinoJob.queryList.queries Dienst: dataproc.googleapis.com REST API: PATCH /v1beta2/projects/{project_id}/regions/{region}/jobs/{job_id} RPC-Methoden: google.cloud.dataproc.v1beta2.JobController.UpdateJob job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value Dienst: dataproc.googleapis.com REST API: POST /v1/projects/{project_id}/regions/{region}/jobs:submit RPC-Methoden: google.cloud.dataproc.v1.JobController.SubmitJob job.flinkJob.args job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pyflinkJob.args job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value job.trinoJob.queryList.queries Dienst: dataproc.googleapis.com REST API: POST /v1/projects/{project_id}/regions/{region}/jobs:submitAsOperation RPC-Methoden: google.cloud.dataproc.v1.JobController.SubmitJobAsOperation job.flinkJob.args job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pyflinkJob.args job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value job.trinoJob.queryList.queries Dienst: dataproc.googleapis.com REST API: POST /v1beta2/projects/{project_id}/regions/{region}/jobs:submit RPC-Methoden: google.cloud.dataproc.v1beta2.JobController.SubmitJob job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value Dienst: dataproc.googleapis.com REST API: POST /v1beta2/projects/{project_id}/regions/{region}/jobs:submitAsOperation RPC-Methoden: google.cloud.dataproc.v1beta2.JobController.SubmitJobAsOperation job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value Ressource: dataproc.googleapis.com/WorkflowTemplate In der folgenden Tabelle sind die API-Ressourcen und -Felder aufgeführt, die für die Verarbeitung von Daten konzipiert sind, die gemäß ITAR durch die Datengrenze geschützt sind. API-Methode Geschützte Felder Dienst: dataproc.googleapis.com REST API: POST /v1/{parent=projects/*/locations/*}/workflowTemplates RPC-Methoden: google.cloud.dataproc.v1.WorkflowTemplateService.CreateWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Dienst: dataproc.googleapis.com REST API: POST /v1/{parent=projects/*/locations/*}/workflowTemplates:instantiateInline RPC-Methoden: google.cloud.dataproc.v1.WorkflowTemplateService.InstantiateInlineWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Dienst: dataproc.googleapis.com REST API: POST /v1/{parent=projects/*/regions/*}/workflowTemplates RPC-Methoden: google.cloud.dataproc.v1.WorkflowTemplateService.CreateWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Dienst: dataproc.googleapis.com REST API: POST /v1/{parent=projects/*/regions/*}/workflowTemplates:instantiateInline RPC-Methoden: google.cloud.dataproc.v1.WorkflowTemplateService.InstantiateInlineWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Dienst: dataproc.googleapis.com REST API: POST /v1beta2/{parent=projects/*/locations/*}/workflowTemplates RPC-Methoden: google.cloud.dataproc.v1beta2.WorkflowTemplateService.CreateWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Dienst: dataproc.googleapis.com REST API: POST /v1beta2/{parent=projects/*/locations/*}/workflowTemplates:instantiateInline RPC-Methoden: google.cloud.dataproc.v1beta2.WorkflowTemplateService.InstantiateInlineWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Dienst: dataproc.googleapis.com REST API: POST /v1beta2/{parent=projects/*/regions/*}/workflowTemplates RPC-Methoden: google.cloud.dataproc.v1beta2.WorkflowTemplateService.CreateWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Dienst: dataproc.googleapis.com REST API: POST /v1beta2/{parent=projects/*/regions/*}/workflowTemplates:instantiateInline RPC-Methoden: google.cloud.dataproc.v1beta2.WorkflowTemplateService.InstantiateInlineWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Dienst: dataproc.googleapis.com REST API: PUT /v1/{template.name=projects/*/locations/*/workflowTemplates/*} RPC-Methoden: google.cloud.dataproc.v1.WorkflowTemplateService.UpdateWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Dienst: dataproc.googleapis.com REST API: PUT /v1/{template.name=projects/*/regions/*/workflowTemplates/*} RPC-Methoden: google.cloud.dataproc.v1.WorkflowTemplateService.UpdateWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Dienst: dataproc.googleapis.com REST API: PUT /v1beta2/{template.name=projects/*/locations/*/workflowTemplates/*} RPC-Methoden: google.cloud.dataproc.v1beta2.WorkflowTemplateService.UpdateWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Dienst: dataproc.googleapis.com REST API: PUT /v1beta2/{template.name=projects/*/regions/*/workflowTemplates/*} RPC-Methoden: google.cloud.dataproc.v1beta2.WorkflowTemplateService.UpdateWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Felder, die nicht für vertrauliche Daten vorgesehen sind Die folgende Tabelle enthält eine beispielhafte Liste von Feldkategorien und spezifischen Feldern, die nicht für vertrauliche Informationen geeignet sind. Um die Compliance aufrechtzuerhalten, sollten Sie keine geschützten Daten in diese Felder einfügen. Eine vollständige Liste erhalten Sie von Ihrem Google Cloud-Ansprechpartner. Kategorie Felder Cluster-Identifizierung cluster clusterName clusterUuid Job-Identifizierung job jobId yarnApplicationId yarnApplicationIds Jobspezifische Konfigurationen job.flinkJob.jarFileUris job.hadoopJob.fileUris job.pigJob.queryFileUri job.pyflinkJob.pythonFileUris job.pysparkJob.fileUris job.sparkSqlJob.queryFileUri Standortidentifizierung region Netzwerk und Verbindung cluster.config.gceClusterConfig.networkUri cluster.config.gceClusterConfig.nodeGroupAffinity.nodeGroupUri cluster.config.gceClusterConfig.subnetworkUri Projektidentifizierung project projectId Ressourcenkonfiguration – Compute cluster.config.auxiliaryNodeGroups.nodeGroup.nodeGroupConfig.machineTypeUri cluster.config.masterConfig.machineTypeUri cluster.config.masterConfig.minCpuPlatform cluster.config.secondaryWorkerConfig.machineTypeUri cluster.config.workerConfig.machineTypeUri cluster.config.workerConfig.minCpuPlatform Ressourcenkonfiguration – Speicher cluster.config.auxiliaryNodeGroups.nodeGroup.nodeGroupConfig.diskConfig.bootDiskType cluster.config.masterConfig.diskConfig.bootDiskType cluster.config.masterConfig.diskConfig.localSsdInterface cluster.config.secondaryWorkerConfig.diskConfig.bootDiskType cluster.config.workerConfig.diskConfig.bootDiskType cluster.config.workerConfig.diskConfig.localSsdInterface Sicherheitskonfiguration – Verschlüsselung cluster.config.encryptionConfig.gcePdKmsKeyName cluster.config.encryptionConfig.kmsKey Sicherheitskonfiguration – Kerberos cluster.config.securityConfig.kerberosConfig.kdcDbKeyUri cluster.config.securityConfig.kerberosConfig.keyPasswordUri cluster.config.securityConfig.kerberosConfig.keystoreUri cluster.config.securityConfig.kerberosConfig.kmsKeyUri cluster.config.securityConfig.kerberosConfig.rootPrincipalPasswordUri cluster.config.securityConfig.kerberosConfig.truststoreUri

Nächste Schritte

• Weitere Informationen zur Compliance in Google Cloud