Neste documento, descrevemos os recursos, as configurações e as APIs do Serviço Gerenciado para Apache Spark que se alinham aos controles dos pacotes de controle compatíveis. Neste documento, presumimos que você esteja usando o Assured Workloads.

Mostrar tudo Limite de dados para ITAR

Limite de dados para ITAR Serviços compatíveis A tabela a seguir lista as APIs e versões do Serviço Gerenciado para Apache Spark que atendem aos requisitos do Limite de Dados para ITAR. Serviço Versão Status dataproc.googleapis.com v1 COMPATÍVEL dataproc.googleapis.com v1beta2 COMPATÍVEL Regiões com suporte para compliance O Serviço Gerenciado para Apache Spark está disponível para o Limite de Dados para ITAR nas seguintes regiões Google Cloud : us-central1 us-central2 us-east1 us-east4 us-east5 us-south1 us-west1 us-west2 us-west3 us-west4 Campos da API para dados sensíveis Recurso: dataproc.googleapis.com/Batch A tabela a seguir especifica os recursos e campos da API projetados para processar dados protegidos pelo Limite de Dados para ITAR. Método de API Campos protegidos Serviço: dataproc.googleapis.com API REST: POST /v1/{parent=projects/*/locations/*}/batches Métodos RPC: google.cloud.dataproc.v1.BatchController.CreateBatch batch.pysparkBatch.args batch.rayBatch.args batch.sparkBatch.args batch.sparkRBatch.args batch.sparkSqlBatch.queryVariables.key batch.sparkSqlBatch.queryVariables.value Recurso: dataproc.googleapis.com/Job A tabela a seguir especifica os recursos e campos da API projetados para processar dados protegidos pelo Limite de Dados para ITAR. Método de API Campos protegidos Serviço: dataproc.googleapis.com API REST: PATCH /v1/projects/{project_id}/regions/{region}/jobs/{job_id} Métodos RPC: google.cloud.dataproc.v1.JobController.UpdateJob job.flinkJob.args job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pyflinkJob.args job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value job.trinoJob.queryList.queries Serviço: dataproc.googleapis.com API REST: PATCH /v1beta2/projects/{project_id}/regions/{region}/jobs/{job_id} Métodos RPC: google.cloud.dataproc.v1beta2.JobController.UpdateJob job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value Serviço: dataproc.googleapis.com API REST: POST /v1/projects/{project_id}/regions/{region}/jobs:submit Métodos RPC: google.cloud.dataproc.v1.JobController.SubmitJob job.flinkJob.args job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pyflinkJob.args job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value job.trinoJob.queryList.queries Serviço: dataproc.googleapis.com API REST: POST /v1/projects/{project_id}/regions/{region}/jobs:submitAsOperation Métodos RPC: google.cloud.dataproc.v1.JobController.SubmitJobAsOperation job.flinkJob.args job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pyflinkJob.args job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value job.trinoJob.queryList.queries Serviço: dataproc.googleapis.com API REST: POST /v1beta2/projects/{project_id}/regions/{region}/jobs:submit Métodos RPC: google.cloud.dataproc.v1beta2.JobController.SubmitJob job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value Serviço: dataproc.googleapis.com API REST: POST /v1beta2/projects/{project_id}/regions/{region}/jobs:submitAsOperation Métodos RPC: google.cloud.dataproc.v1beta2.JobController.SubmitJobAsOperation job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value Recurso: dataproc.googleapis.com/WorkflowTemplate A tabela a seguir especifica os recursos e campos da API projetados para processar dados protegidos pelo Limite de Dados para ITAR. Método de API Campos protegidos Serviço: dataproc.googleapis.com API REST: POST /v1/{parent=projects/*/locations/*}/workflowTemplates Métodos RPC: google.cloud.dataproc.v1.WorkflowTemplateService.CreateWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Serviço: dataproc.googleapis.com API REST: POST /v1/{parent=projects/*/locations/*}/workflowTemplates:instantiateInline Métodos RPC: google.cloud.dataproc.v1.WorkflowTemplateService.InstantiateInlineWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Serviço: dataproc.googleapis.com API REST: POST /v1/{parent=projects/*/regions/*}/workflowTemplates Métodos RPC: google.cloud.dataproc.v1.WorkflowTemplateService.CreateWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Serviço: dataproc.googleapis.com API REST: POST /v1/{parent=projects/*/regions/*}/workflowTemplates:instantiateInline Métodos RPC: google.cloud.dataproc.v1.WorkflowTemplateService.InstantiateInlineWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Serviço: dataproc.googleapis.com API REST: POST /v1beta2/{parent=projects/*/locations/*}/workflowTemplates Métodos RPC: google.cloud.dataproc.v1beta2.WorkflowTemplateService.CreateWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Serviço: dataproc.googleapis.com API REST: POST /v1beta2/{parent=projects/*/locations/*}/workflowTemplates:instantiateInline Métodos RPC: google.cloud.dataproc.v1beta2.WorkflowTemplateService.InstantiateInlineWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Serviço: dataproc.googleapis.com API REST: POST /v1beta2/{parent=projects/*/regions/*}/workflowTemplates Métodos RPC: google.cloud.dataproc.v1beta2.WorkflowTemplateService.CreateWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Serviço: dataproc.googleapis.com API REST: POST /v1beta2/{parent=projects/*/regions/*}/workflowTemplates:instantiateInline Métodos RPC: google.cloud.dataproc.v1beta2.WorkflowTemplateService.InstantiateInlineWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Serviço: dataproc.googleapis.com API REST: PUT /v1/{template.name=projects/*/locations/*/workflowTemplates/*} Métodos RPC: google.cloud.dataproc.v1.WorkflowTemplateService.UpdateWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Serviço: dataproc.googleapis.com API REST: PUT /v1/{template.name=projects/*/regions/*/workflowTemplates/*} Métodos RPC: google.cloud.dataproc.v1.WorkflowTemplateService.UpdateWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Serviço: dataproc.googleapis.com API REST: PUT /v1beta2/{template.name=projects/*/locations/*/workflowTemplates/*} Métodos RPC: google.cloud.dataproc.v1beta2.WorkflowTemplateService.UpdateWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Serviço: dataproc.googleapis.com API REST: PUT /v1beta2/{template.name=projects/*/regions/*/workflowTemplates/*} Métodos RPC: google.cloud.dataproc.v1beta2.WorkflowTemplateService.UpdateWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Campos não destinados a dados sensíveis A tabela a seguir mostra uma lista ilustrativa de categorias e campos específicos que não são adequados para informações sensíveis. Para manter a compliance, evite colocar dados protegidos nesses campos. Para uma lista completa, entre em contato com seu representante do Google Cloud. Categoria Campos Identificação de cluster cluster clusterName clusterUuid Identificação do job job jobId yarnApplicationId yarnApplicationIds Configurações específicas do job job.flinkJob.jarFileUris job.hadoopJob.fileUris job.pigJob.queryFileUri job.pyflinkJob.pythonFileUris job.pysparkJob.fileUris job.sparkSqlJob.queryFileUri Identificação de local region Rede e conectividade cluster.config.gceClusterConfig.networkUri cluster.config.gceClusterConfig.nodeGroupAffinity.nodeGroupUri cluster.config.gceClusterConfig.subnetworkUri Identificação do projeto project projectId Configuração de recursos: computação cluster.config.auxiliaryNodeGroups.nodeGroup.nodeGroupConfig.machineTypeUri cluster.config.masterConfig.machineTypeUri cluster.config.masterConfig.minCpuPlatform cluster.config.secondaryWorkerConfig.machineTypeUri cluster.config.workerConfig.machineTypeUri cluster.config.workerConfig.minCpuPlatform Configuração de recursos: armazenamento cluster.config.auxiliaryNodeGroups.nodeGroup.nodeGroupConfig.diskConfig.bootDiskType cluster.config.masterConfig.diskConfig.bootDiskType cluster.config.masterConfig.diskConfig.localSsdInterface cluster.config.secondaryWorkerConfig.diskConfig.bootDiskType cluster.config.workerConfig.diskConfig.bootDiskType cluster.config.workerConfig.diskConfig.localSsdInterface Configuração de segurança: criptografia cluster.config.encryptionConfig.gcePdKmsKeyName cluster.config.encryptionConfig.kmsKey Configuração de segurança - Kerberos cluster.config.securityConfig.kerberosConfig.kdcDbKeyUri cluster.config.securityConfig.kerberosConfig.keyPasswordUri cluster.config.securityConfig.kerberosConfig.keystoreUri cluster.config.securityConfig.kerberosConfig.kmsKeyUri cluster.config.securityConfig.kerberosConfig.rootPrincipalPasswordUri cluster.config.securityConfig.kerberosConfig.truststoreUri

A seguir

• Saiba mais sobre compliance em Google Cloud.