En este documento, se describen las funciones, las configuraciones y las APIs de Managed Service para Apache Spark que se alinean con los controles de los paquetes de control admitidos. En este documento, se supone que usas Assured Workloads.

Límite de datos para ITAR

Servicios compatibles

En la siguiente tabla, se enumeran las APIs y las versiones de Managed Service para Apache Spark que cumplen con los requisitos del límite de datos para la ITAR.

Servicio Versión Estado
dataproc.googleapis.com v1 ADMITIDO
dataproc.googleapis.com v1beta2 ADMITIDO

Regiones admitidas para el cumplimiento

Managed Service para Apache Spark está disponible para el límite de datos del ITAR en las siguientes Google Cloud regiones:

  • us-central1
  • us-central2
  • us-east1
  • us-east4
  • us-east5
  • us-south1
  • us-west1
  • us-west2
  • us-west3
  • us-west4

Campos de la API para datos sensibles

Recurso: dataproc.googleapis.com/Batch

En la siguiente tabla, se especifican los recursos y campos de la API diseñados para controlar los datos protegidos en virtud del límite de datos para la ITAR.

Método de API Campos protegidos

Servicio: dataproc.googleapis.com

API de REST: POST /v1/{parent=projects/*/locations/*}/batches

Métodos RPC:

  • google.cloud.dataproc.v1.BatchController.CreateBatch
  • batch.pysparkBatch.args
  • batch.rayBatch.args
  • batch.sparkBatch.args
  • batch.sparkRBatch.args
  • batch.sparkSqlBatch.queryVariables.key
  • batch.sparkSqlBatch.queryVariables.value

Recurso: dataproc.googleapis.com/Job

En la siguiente tabla, se especifican los recursos y campos de la API diseñados para controlar los datos protegidos en virtud del límite de datos para la ITAR.

Método de API Campos protegidos

Servicio: dataproc.googleapis.com

API de REST: PATCH /v1/projects/{project_id}/regions/{region}/jobs/{job_id}

Métodos RPC:

  • google.cloud.dataproc.v1.JobController.UpdateJob
  • job.flinkJob.args
  • job.hadoopJob.args
  • job.hiveJob.queryList.queries
  • job.hiveJob.scriptVariables.key
  • job.hiveJob.scriptVariables.value
  • job.pigJob.queryList.queries
  • job.pigJob.scriptVariables.key
  • job.pigJob.scriptVariables.value
  • job.prestoJob.queryList.queries
  • job.pyflinkJob.args
  • job.pysparkJob.args
  • job.sparkJob.args
  • job.sparkRJob.args
  • job.sparkSqlJob.queryList.queries
  • job.sparkSqlJob.scriptVariables.key
  • job.sparkSqlJob.scriptVariables.value
  • job.trinoJob.queryList.queries

Servicio: dataproc.googleapis.com

API de REST: PATCH /v1beta2/projects/{project_id}/regions/{region}/jobs/{job_id}

Métodos RPC:

  • google.cloud.dataproc.v1beta2.JobController.UpdateJob
  • job.hadoopJob.args
  • job.hiveJob.queryList.queries
  • job.hiveJob.scriptVariables.key
  • job.hiveJob.scriptVariables.value
  • job.pigJob.queryList.queries
  • job.pigJob.scriptVariables.key
  • job.pigJob.scriptVariables.value
  • job.prestoJob.queryList.queries
  • job.pysparkJob.args
  • job.sparkJob.args
  • job.sparkRJob.args
  • job.sparkSqlJob.queryList.queries
  • job.sparkSqlJob.scriptVariables.key
  • job.sparkSqlJob.scriptVariables.value

Servicio: dataproc.googleapis.com

API de REST: POST /v1/projects/{project_id}/regions/{region}/jobs:submit

Métodos RPC:

  • google.cloud.dataproc.v1.JobController.SubmitJob
  • job.flinkJob.args
  • job.hadoopJob.args
  • job.hiveJob.queryList.queries
  • job.hiveJob.scriptVariables.key
  • job.hiveJob.scriptVariables.value
  • job.pigJob.queryList.queries
  • job.pigJob.scriptVariables.key
  • job.pigJob.scriptVariables.value
  • job.prestoJob.queryList.queries
  • job.pyflinkJob.args
  • job.pysparkJob.args
  • job.sparkJob.args
  • job.sparkRJob.args
  • job.sparkSqlJob.queryList.queries
  • job.sparkSqlJob.scriptVariables.key
  • job.sparkSqlJob.scriptVariables.value
  • job.trinoJob.queryList.queries

Servicio: dataproc.googleapis.com

API de REST: POST /v1/projects/{project_id}/regions/{region}/jobs:submitAsOperation

Métodos RPC:

  • google.cloud.dataproc.v1.JobController.SubmitJobAsOperation
  • job.flinkJob.args
  • job.hadoopJob.args
  • job.hiveJob.queryList.queries
  • job.hiveJob.scriptVariables.key
  • job.hiveJob.scriptVariables.value
  • job.pigJob.queryList.queries
  • job.pigJob.scriptVariables.key
  • job.pigJob.scriptVariables.value
  • job.prestoJob.queryList.queries
  • job.pyflinkJob.args
  • job.pysparkJob.args
  • job.sparkJob.args
  • job.sparkRJob.args
  • job.sparkSqlJob.queryList.queries
  • job.sparkSqlJob.scriptVariables.key
  • job.sparkSqlJob.scriptVariables.value
  • job.trinoJob.queryList.queries

Servicio: dataproc.googleapis.com

API de REST: POST /v1beta2/projects/{project_id}/regions/{region}/jobs:submit

Métodos RPC:

  • google.cloud.dataproc.v1beta2.JobController.SubmitJob
  • job.hadoopJob.args
  • job.hiveJob.queryList.queries
  • job.hiveJob.scriptVariables.key
  • job.hiveJob.scriptVariables.value
  • job.pigJob.queryList.queries
  • job.pigJob.scriptVariables.key
  • job.pigJob.scriptVariables.value
  • job.prestoJob.queryList.queries
  • job.pysparkJob.args
  • job.sparkJob.args
  • job.sparkRJob.args
  • job.sparkSqlJob.queryList.queries
  • job.sparkSqlJob.scriptVariables.key
  • job.sparkSqlJob.scriptVariables.value

Servicio: dataproc.googleapis.com

API de REST: POST /v1beta2/projects/{project_id}/regions/{region}/jobs:submitAsOperation

Métodos RPC:

  • google.cloud.dataproc.v1beta2.JobController.SubmitJobAsOperation
  • job.hadoopJob.args
  • job.hiveJob.queryList.queries
  • job.hiveJob.scriptVariables.key
  • job.hiveJob.scriptVariables.value
  • job.pigJob.queryList.queries
  • job.pigJob.scriptVariables.key
  • job.pigJob.scriptVariables.value
  • job.prestoJob.queryList.queries
  • job.pysparkJob.args
  • job.sparkJob.args
  • job.sparkRJob.args
  • job.sparkSqlJob.queryList.queries
  • job.sparkSqlJob.scriptVariables.key
  • job.sparkSqlJob.scriptVariables.value

Recurso: dataproc.googleapis.com/WorkflowTemplate

En la siguiente tabla, se especifican los recursos y campos de la API diseñados para controlar los datos protegidos en virtud del límite de datos para la ITAR.

Método de API Campos protegidos

Servicio: dataproc.googleapis.com

API de REST: POST /v1/{parent=projects/*/locations/*}/workflowTemplates

Métodos RPC:

  • google.cloud.dataproc.v1.WorkflowTemplateService.CreateWorkflowTemplate
  • template.jobs.flinkJob.args
  • template.jobs.hadoopJob.args
  • template.jobs.hiveJob.queryList.queries
  • template.jobs.hiveJob.scriptVariables.key
  • template.jobs.hiveJob.scriptVariables.value
  • template.jobs.pigJob.queryList.queries
  • template.jobs.pigJob.scriptVariables.key
  • template.jobs.pigJob.scriptVariables.value
  • template.jobs.prestoJob.queryList.queries
  • template.jobs.pyflinkJob.args
  • template.jobs.pysparkJob.args
  • template.jobs.sparkJob.args
  • template.jobs.sparkRJob.args
  • template.jobs.sparkSqlJob.queryList.queries
  • template.jobs.sparkSqlJob.scriptVariables.key
  • template.jobs.sparkSqlJob.scriptVariables.value
  • template.jobs.trinoJob.queryList.queries

Servicio: dataproc.googleapis.com

API de REST: POST /v1/{parent=projects/*/locations/*}/workflowTemplates:instantiateInline

Métodos RPC:

  • google.cloud.dataproc.v1.WorkflowTemplateService.InstantiateInlineWorkflowTemplate
  • template.jobs.flinkJob.args
  • template.jobs.hadoopJob.args
  • template.jobs.hiveJob.queryList.queries
  • template.jobs.hiveJob.scriptVariables.key
  • template.jobs.hiveJob.scriptVariables.value
  • template.jobs.pigJob.queryList.queries
  • template.jobs.pigJob.scriptVariables.key
  • template.jobs.pigJob.scriptVariables.value
  • template.jobs.prestoJob.queryList.queries
  • template.jobs.pyflinkJob.args
  • template.jobs.pysparkJob.args
  • template.jobs.sparkJob.args
  • template.jobs.sparkRJob.args
  • template.jobs.sparkSqlJob.queryList.queries
  • template.jobs.sparkSqlJob.scriptVariables.key
  • template.jobs.sparkSqlJob.scriptVariables.value
  • template.jobs.trinoJob.queryList.queries

Servicio: dataproc.googleapis.com

API de REST: POST /v1/{parent=projects/*/regions/*}/workflowTemplates

Métodos RPC:

  • google.cloud.dataproc.v1.WorkflowTemplateService.CreateWorkflowTemplate
  • template.jobs.flinkJob.args
  • template.jobs.hadoopJob.args
  • template.jobs.hiveJob.queryList.queries
  • template.jobs.hiveJob.scriptVariables.key
  • template.jobs.hiveJob.scriptVariables.value
  • template.jobs.pigJob.queryList.queries
  • template.jobs.pigJob.scriptVariables.key
  • template.jobs.pigJob.scriptVariables.value
  • template.jobs.prestoJob.queryList.queries
  • template.jobs.pyflinkJob.args
  • template.jobs.pysparkJob.args
  • template.jobs.sparkJob.args
  • template.jobs.sparkRJob.args
  • template.jobs.sparkSqlJob.queryList.queries
  • template.jobs.sparkSqlJob.scriptVariables.key
  • template.jobs.sparkSqlJob.scriptVariables.value
  • template.jobs.trinoJob.queryList.queries

Servicio: dataproc.googleapis.com

API de REST: POST /v1/{parent=projects/*/regions/*}/workflowTemplates:instantiateInline

Métodos RPC:

  • google.cloud.dataproc.v1.WorkflowTemplateService.InstantiateInlineWorkflowTemplate
  • template.jobs.flinkJob.args
  • template.jobs.hadoopJob.args
  • template.jobs.hiveJob.queryList.queries
  • template.jobs.hiveJob.scriptVariables.key
  • template.jobs.hiveJob.scriptVariables.value
  • template.jobs.pigJob.queryList.queries
  • template.jobs.pigJob.scriptVariables.key
  • template.jobs.pigJob.scriptVariables.value
  • template.jobs.prestoJob.queryList.queries
  • template.jobs.pyflinkJob.args
  • template.jobs.pysparkJob.args
  • template.jobs.sparkJob.args
  • template.jobs.sparkRJob.args
  • template.jobs.sparkSqlJob.queryList.queries
  • template.jobs.sparkSqlJob.scriptVariables.key
  • template.jobs.sparkSqlJob.scriptVariables.value
  • template.jobs.trinoJob.queryList.queries

Servicio: dataproc.googleapis.com

API de REST: POST /v1beta2/{parent=projects/*/locations/*}/workflowTemplates

Métodos RPC:

  • google.cloud.dataproc.v1beta2.WorkflowTemplateService.CreateWorkflowTemplate
  • template.jobs.hadoopJob.args
  • template.jobs.hiveJob.queryList.queries
  • template.jobs.hiveJob.scriptVariables.key
  • template.jobs.hiveJob.scriptVariables.value
  • template.jobs.pigJob.queryList.queries
  • template.jobs.pigJob.scriptVariables.key
  • template.jobs.pigJob.scriptVariables.value
  • template.jobs.prestoJob.queryList.queries
  • template.jobs.pysparkJob.args
  • template.jobs.sparkJob.args
  • template.jobs.sparkRJob.args
  • template.jobs.sparkSqlJob.queryList.queries
  • template.jobs.sparkSqlJob.scriptVariables.key
  • template.jobs.sparkSqlJob.scriptVariables.value

Servicio: dataproc.googleapis.com

API de REST: POST /v1beta2/{parent=projects/*/locations/*}/workflowTemplates:instantiateInline

Métodos RPC:

  • google.cloud.dataproc.v1beta2.WorkflowTemplateService.InstantiateInlineWorkflowTemplate
  • template.jobs.hadoopJob.args
  • template.jobs.hiveJob.queryList.queries
  • template.jobs.hiveJob.scriptVariables.key
  • template.jobs.hiveJob.scriptVariables.value
  • template.jobs.pigJob.queryList.queries
  • template.jobs.pigJob.scriptVariables.key
  • template.jobs.pigJob.scriptVariables.value
  • template.jobs.prestoJob.queryList.queries
  • template.jobs.pysparkJob.args
  • template.jobs.sparkJob.args
  • template.jobs.sparkRJob.args
  • template.jobs.sparkSqlJob.queryList.queries
  • template.jobs.sparkSqlJob.scriptVariables.key
  • template.jobs.sparkSqlJob.scriptVariables.value

Servicio: dataproc.googleapis.com

API de REST: POST /v1beta2/{parent=projects/*/regions/*}/workflowTemplates

Métodos RPC:

  • google.cloud.dataproc.v1beta2.WorkflowTemplateService.CreateWorkflowTemplate
  • template.jobs.hadoopJob.args
  • template.jobs.hiveJob.queryList.queries
  • template.jobs.hiveJob.scriptVariables.key
  • template.jobs.hiveJob.scriptVariables.value
  • template.jobs.pigJob.queryList.queries
  • template.jobs.pigJob.scriptVariables.key
  • template.jobs.pigJob.scriptVariables.value
  • template.jobs.prestoJob.queryList.queries
  • template.jobs.pysparkJob.args
  • template.jobs.sparkJob.args
  • template.jobs.sparkRJob.args
  • template.jobs.sparkSqlJob.queryList.queries
  • template.jobs.sparkSqlJob.scriptVariables.key
  • template.jobs.sparkSqlJob.scriptVariables.value

Servicio: dataproc.googleapis.com

API de REST: POST /v1beta2/{parent=projects/*/regions/*}/workflowTemplates:instantiateInline

Métodos RPC:

  • google.cloud.dataproc.v1beta2.WorkflowTemplateService.InstantiateInlineWorkflowTemplate
  • template.jobs.hadoopJob.args
  • template.jobs.hiveJob.queryList.queries
  • template.jobs.hiveJob.scriptVariables.key
  • template.jobs.hiveJob.scriptVariables.value
  • template.jobs.pigJob.queryList.queries
  • template.jobs.pigJob.scriptVariables.key
  • template.jobs.pigJob.scriptVariables.value
  • template.jobs.prestoJob.queryList.queries
  • template.jobs.pysparkJob.args
  • template.jobs.sparkJob.args
  • template.jobs.sparkRJob.args
  • template.jobs.sparkSqlJob.queryList.queries
  • template.jobs.sparkSqlJob.scriptVariables.key
  • template.jobs.sparkSqlJob.scriptVariables.value

Servicio: dataproc.googleapis.com

API de REST: PUT /v1/{template.name=projects/*/locations/*/workflowTemplates/*}

Métodos RPC:

  • google.cloud.dataproc.v1.WorkflowTemplateService.UpdateWorkflowTemplate
  • template.jobs.flinkJob.args
  • template.jobs.hadoopJob.args
  • template.jobs.hiveJob.queryList.queries
  • template.jobs.hiveJob.scriptVariables.key
  • template.jobs.hiveJob.scriptVariables.value
  • template.jobs.pigJob.queryList.queries
  • template.jobs.pigJob.scriptVariables.key
  • template.jobs.pigJob.scriptVariables.value
  • template.jobs.prestoJob.queryList.queries
  • template.jobs.pyflinkJob.args
  • template.jobs.pysparkJob.args
  • template.jobs.sparkJob.args
  • template.jobs.sparkRJob.args
  • template.jobs.sparkSqlJob.queryList.queries
  • template.jobs.sparkSqlJob.scriptVariables.key
  • template.jobs.sparkSqlJob.scriptVariables.value
  • template.jobs.trinoJob.queryList.queries

Servicio: dataproc.googleapis.com

API de REST: PUT /v1/{template.name=projects/*/regions/*/workflowTemplates/*}

Métodos RPC:

  • google.cloud.dataproc.v1.WorkflowTemplateService.UpdateWorkflowTemplate
  • template.jobs.flinkJob.args
  • template.jobs.hadoopJob.args
  • template.jobs.hiveJob.queryList.queries
  • template.jobs.hiveJob.scriptVariables.key
  • template.jobs.hiveJob.scriptVariables.value
  • template.jobs.pigJob.queryList.queries
  • template.jobs.pigJob.scriptVariables.key
  • template.jobs.pigJob.scriptVariables.value
  • template.jobs.prestoJob.queryList.queries
  • template.jobs.pyflinkJob.args
  • template.jobs.pysparkJob.args
  • template.jobs.sparkJob.args
  • template.jobs.sparkRJob.args
  • template.jobs.sparkSqlJob.queryList.queries
  • template.jobs.sparkSqlJob.scriptVariables.key
  • template.jobs.sparkSqlJob.scriptVariables.value
  • template.jobs.trinoJob.queryList.queries

Servicio: dataproc.googleapis.com

API de REST: PUT /v1beta2/{template.name=projects/*/locations/*/workflowTemplates/*}

Métodos RPC:

  • google.cloud.dataproc.v1beta2.WorkflowTemplateService.UpdateWorkflowTemplate
  • template.jobs.hadoopJob.args
  • template.jobs.hiveJob.queryList.queries
  • template.jobs.hiveJob.scriptVariables.key
  • template.jobs.hiveJob.scriptVariables.value
  • template.jobs.pigJob.queryList.queries
  • template.jobs.pigJob.scriptVariables.key
  • template.jobs.pigJob.scriptVariables.value
  • template.jobs.prestoJob.queryList.queries
  • template.jobs.pysparkJob.args
  • template.jobs.sparkJob.args
  • template.jobs.sparkRJob.args
  • template.jobs.sparkSqlJob.queryList.queries
  • template.jobs.sparkSqlJob.scriptVariables.key
  • template.jobs.sparkSqlJob.scriptVariables.value

Servicio: dataproc.googleapis.com

API de REST: PUT /v1beta2/{template.name=projects/*/regions/*/workflowTemplates/*}

Métodos RPC:

  • google.cloud.dataproc.v1beta2.WorkflowTemplateService.UpdateWorkflowTemplate
  • template.jobs.hadoopJob.args
  • template.jobs.hiveJob.queryList.queries
  • template.jobs.hiveJob.scriptVariables.key
  • template.jobs.hiveJob.scriptVariables.value
  • template.jobs.pigJob.queryList.queries
  • template.jobs.pigJob.scriptVariables.key
  • template.jobs.pigJob.scriptVariables.value
  • template.jobs.prestoJob.queryList.queries
  • template.jobs.pysparkJob.args
  • template.jobs.sparkJob.args
  • template.jobs.sparkRJob.args
  • template.jobs.sparkSqlJob.queryList.queries
  • template.jobs.sparkSqlJob.scriptVariables.key
  • template.jobs.sparkSqlJob.scriptVariables.value

Campos no diseñados para datos sensibles

En la siguiente tabla, se proporciona una lista ilustrativa de categorías de campos y campos específicos que no son adecuados para la información sensible. Para mantener el cumplimiento, evita colocar datos protegidos en estos campos. Para obtener una lista completa, comunícate con tu representante de Google Cloud.

Categoría Campos
Identificación de clústeres
  • cluster
  • clusterName
  • clusterUuid
Identificación del trabajo
  • job
  • jobId
  • yarnApplicationId
  • yarnApplicationIds
Configuraciones específicas del trabajo
  • job.flinkJob.jarFileUris
  • job.hadoopJob.fileUris
  • job.pigJob.queryFileUri
  • job.pyflinkJob.pythonFileUris
  • job.pysparkJob.fileUris
  • job.sparkSqlJob.queryFileUri
Identificación de la ubicación
  • region
Red y conectividad
  • cluster.config.gceClusterConfig.networkUri
  • cluster.config.gceClusterConfig.nodeGroupAffinity.nodeGroupUri
  • cluster.config.gceClusterConfig.subnetworkUri
Identificación del proyecto
  • project
  • projectId
Configuración de recursos: procesamiento
  • cluster.config.auxiliaryNodeGroups.nodeGroup.nodeGroupConfig.machineTypeUri
  • cluster.config.masterConfig.machineTypeUri
  • cluster.config.masterConfig.minCpuPlatform
  • cluster.config.secondaryWorkerConfig.machineTypeUri
  • cluster.config.workerConfig.machineTypeUri
  • cluster.config.workerConfig.minCpuPlatform
Configuración de recursos: almacenamiento
  • cluster.config.auxiliaryNodeGroups.nodeGroup.nodeGroupConfig.diskConfig.bootDiskType
  • cluster.config.masterConfig.diskConfig.bootDiskType
  • cluster.config.masterConfig.diskConfig.localSsdInterface
  • cluster.config.secondaryWorkerConfig.diskConfig.bootDiskType
  • cluster.config.workerConfig.diskConfig.bootDiskType
  • cluster.config.workerConfig.diskConfig.localSsdInterface
Configuración de seguridad: encriptación
  • cluster.config.encryptionConfig.gcePdKmsKeyName
  • cluster.config.encryptionConfig.kmsKey
Configuración de seguridad: Kerberos
  • cluster.config.securityConfig.kerberosConfig.kdcDbKeyUri
  • cluster.config.securityConfig.kerberosConfig.keyPasswordUri
  • cluster.config.securityConfig.kerberosConfig.keystoreUri
  • cluster.config.securityConfig.kerberosConfig.kmsKeyUri
  • cluster.config.securityConfig.kerberosConfig.rootPrincipalPasswordUri
  • cluster.config.securityConfig.kerberosConfig.truststoreUri

¿Qué sigue?