Dokumen ini menjelaskan fitur, konfigurasi, dan API di Managed Service untuk Apache Spark yang sesuai dengan kontrol untuk paket kontrol yang didukung. Dokumen ini mengasumsikan bahwa Anda menggunakan Assured Workloads.

Tampilkan Semua Batas Data untuk ITAR

Batas Data untuk ITAR Layanan yang didukung Tabel berikut mencantumkan API dan versi Managed Service untuk Apache Spark yang memenuhi persyaratan Data Boundary untuk ITAR. Layanan Versi Status dataproc.googleapis.com v1 DIDUKUNG dataproc.googleapis.com v1beta2 DIDUKUNG Wilayah yang mendukung kepatuhan Managed Service untuk Apache Spark tersedia untuk Data Boundary untuk ITAR di wilayah Google Cloud berikut: us-central1 us-central2 us-east1 us-east4 us-east5 us-south1 us-west1 us-west2 us-west3 us-west4 Kolom API untuk data sensitif Resource: dataproc.googleapis.com/Batch Tabel berikut menentukan resource dan kolom API yang dirancang untuk menangani data yang dilindungi berdasarkan Data Boundary untuk ITAR. Metode API Kolom yang dilindungi Layanan: dataproc.googleapis.com REST API: POST /v1/{parent=projects/*/locations/*}/batches Metode RPC: google.cloud.dataproc.v1.BatchController.CreateBatch batch.pysparkBatch.args batch.rayBatch.args batch.sparkBatch.args batch.sparkRBatch.args batch.sparkSqlBatch.queryVariables.key batch.sparkSqlBatch.queryVariables.value Resource: dataproc.googleapis.com/Job Tabel berikut menentukan resource dan kolom API yang dirancang untuk menangani data yang dilindungi berdasarkan Data Boundary untuk ITAR. Metode API Kolom yang dilindungi Layanan: dataproc.googleapis.com REST API: PATCH /v1/projects/{project_id}/regions/{region}/jobs/{job_id} Metode RPC: google.cloud.dataproc.v1.JobController.UpdateJob job.flinkJob.args job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pyflinkJob.args job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value job.trinoJob.queryList.queries Layanan: dataproc.googleapis.com REST API: PATCH /v1beta2/projects/{project_id}/regions/{region}/jobs/{job_id} Metode RPC: google.cloud.dataproc.v1beta2.JobController.UpdateJob job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value Layanan: dataproc.googleapis.com REST API: POST /v1/projects/{project_id}/regions/{region}/jobs:submit Metode RPC: google.cloud.dataproc.v1.JobController.SubmitJob job.flinkJob.args job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pyflinkJob.args job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value job.trinoJob.queryList.queries Layanan: dataproc.googleapis.com REST API: POST /v1/projects/{project_id}/regions/{region}/jobs:submitAsOperation Metode RPC: google.cloud.dataproc.v1.JobController.SubmitJobAsOperation job.flinkJob.args job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pyflinkJob.args job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value job.trinoJob.queryList.queries Layanan: dataproc.googleapis.com REST API: POST /v1beta2/projects/{project_id}/regions/{region}/jobs:submit Metode RPC: google.cloud.dataproc.v1beta2.JobController.SubmitJob job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value Layanan: dataproc.googleapis.com REST API: POST /v1beta2/projects/{project_id}/regions/{region}/jobs:submitAsOperation Metode RPC: google.cloud.dataproc.v1beta2.JobController.SubmitJobAsOperation job.hadoopJob.args job.hiveJob.queryList.queries job.hiveJob.scriptVariables.key job.hiveJob.scriptVariables.value job.pigJob.queryList.queries job.pigJob.scriptVariables.key job.pigJob.scriptVariables.value job.prestoJob.queryList.queries job.pysparkJob.args job.sparkJob.args job.sparkRJob.args job.sparkSqlJob.queryList.queries job.sparkSqlJob.scriptVariables.key job.sparkSqlJob.scriptVariables.value Resource: dataproc.googleapis.com/WorkflowTemplate Tabel berikut menentukan resource dan kolom API yang dirancang untuk menangani data yang dilindungi berdasarkan Data Boundary untuk ITAR. Metode API Kolom yang dilindungi Layanan: dataproc.googleapis.com REST API: POST /v1/{parent=projects/*/locations/*}/workflowTemplates Metode RPC: google.cloud.dataproc.v1.WorkflowTemplateService.CreateWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Layanan: dataproc.googleapis.com REST API: POST /v1/{parent=projects/*/locations/*}/workflowTemplates:instantiateInline Metode RPC: google.cloud.dataproc.v1.WorkflowTemplateService.InstantiateInlineWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Layanan: dataproc.googleapis.com REST API: POST /v1/{parent=projects/*/regions/*}/workflowTemplates Metode RPC: google.cloud.dataproc.v1.WorkflowTemplateService.CreateWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Layanan: dataproc.googleapis.com REST API: POST /v1/{parent=projects/*/regions/*}/workflowTemplates:instantiateInline Metode RPC: google.cloud.dataproc.v1.WorkflowTemplateService.InstantiateInlineWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Layanan: dataproc.googleapis.com REST API: POST /v1beta2/{parent=projects/*/locations/*}/workflowTemplates Metode RPC: google.cloud.dataproc.v1beta2.WorkflowTemplateService.CreateWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Layanan: dataproc.googleapis.com REST API: POST /v1beta2/{parent=projects/*/locations/*}/workflowTemplates:instantiateInline Metode RPC: google.cloud.dataproc.v1beta2.WorkflowTemplateService.InstantiateInlineWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Layanan: dataproc.googleapis.com REST API: POST /v1beta2/{parent=projects/*/regions/*}/workflowTemplates Metode RPC: google.cloud.dataproc.v1beta2.WorkflowTemplateService.CreateWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Layanan: dataproc.googleapis.com REST API: POST /v1beta2/{parent=projects/*/regions/*}/workflowTemplates:instantiateInline Metode RPC: google.cloud.dataproc.v1beta2.WorkflowTemplateService.InstantiateInlineWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Layanan: dataproc.googleapis.com REST API: PUT /v1/{template.name=projects/*/locations/*/workflowTemplates/*} Metode RPC: google.cloud.dataproc.v1.WorkflowTemplateService.UpdateWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Layanan: dataproc.googleapis.com REST API: PUT /v1/{template.name=projects/*/regions/*/workflowTemplates/*} Metode RPC: google.cloud.dataproc.v1.WorkflowTemplateService.UpdateWorkflowTemplate template.jobs.flinkJob.args template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pyflinkJob.args template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value template.jobs.trinoJob.queryList.queries Layanan: dataproc.googleapis.com REST API: PUT /v1beta2/{template.name=projects/*/locations/*/workflowTemplates/*} Metode RPC: google.cloud.dataproc.v1beta2.WorkflowTemplateService.UpdateWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Layanan: dataproc.googleapis.com REST API: PUT /v1beta2/{template.name=projects/*/regions/*/workflowTemplates/*} Metode RPC: google.cloud.dataproc.v1beta2.WorkflowTemplateService.UpdateWorkflowTemplate template.jobs.hadoopJob.args template.jobs.hiveJob.queryList.queries template.jobs.hiveJob.scriptVariables.key template.jobs.hiveJob.scriptVariables.value template.jobs.pigJob.queryList.queries template.jobs.pigJob.scriptVariables.key template.jobs.pigJob.scriptVariables.value template.jobs.prestoJob.queryList.queries template.jobs.pysparkJob.args template.jobs.sparkJob.args template.jobs.sparkRJob.args template.jobs.sparkSqlJob.queryList.queries template.jobs.sparkSqlJob.scriptVariables.key template.jobs.sparkSqlJob.scriptVariables.value Kolom yang tidak ditujukan untuk Data sensitif Tabel berikut memberikan daftar ilustratif kategori kolom dan kolom tertentu yang tidak sesuai untuk informasi sensitif. Untuk mempertahankan kepatuhan, hindari menempatkan data yang dilindungi di kolom ini. Untuk mengetahui daftar lengkapnya, hubungi perwakilan Google Cloud Anda. Kategori Kolom Identifikasi cluster cluster clusterName clusterUuid Identifikasi tugas job jobId yarnApplicationId yarnApplicationIds Konfigurasi khusus tugas job.flinkJob.jarFileUris job.hadoopJob.fileUris job.pigJob.queryFileUri job.pyflinkJob.pythonFileUris job.pysparkJob.fileUris job.sparkSqlJob.queryFileUri Identifikasi lokasi region Jaringan dan konektivitas cluster.config.gceClusterConfig.networkUri cluster.config.gceClusterConfig.nodeGroupAffinity.nodeGroupUri cluster.config.gceClusterConfig.subnetworkUri Identifikasi project project projectId Konfigurasi resource - komputasi cluster.config.auxiliaryNodeGroups.nodeGroup.nodeGroupConfig.machineTypeUri cluster.config.masterConfig.machineTypeUri cluster.config.masterConfig.minCpuPlatform cluster.config.secondaryWorkerConfig.machineTypeUri cluster.config.workerConfig.machineTypeUri cluster.config.workerConfig.minCpuPlatform Konfigurasi resource - penyimpanan cluster.config.auxiliaryNodeGroups.nodeGroup.nodeGroupConfig.diskConfig.bootDiskType cluster.config.masterConfig.diskConfig.bootDiskType cluster.config.masterConfig.diskConfig.localSsdInterface cluster.config.secondaryWorkerConfig.diskConfig.bootDiskType cluster.config.workerConfig.diskConfig.bootDiskType cluster.config.workerConfig.diskConfig.localSsdInterface Konfigurasi keamanan - enkripsi cluster.config.encryptionConfig.gcePdKmsKeyName cluster.config.encryptionConfig.kmsKey Konfigurasi keamanan - kerberos cluster.config.securityConfig.kerberosConfig.kdcDbKeyUri cluster.config.securityConfig.kerberosConfig.keyPasswordUri cluster.config.securityConfig.kerberosConfig.keystoreUri cluster.config.securityConfig.kerberosConfig.kmsKeyUri cluster.config.securityConfig.kerberosConfig.rootPrincipalPasswordUri cluster.config.securityConfig.kerberosConfig.truststoreUri

Langkah berikutnya

• Pelajari lebih lanjut kepatuhan di Google Cloud.