Google 會運用 AI 技術將內容翻譯成你偏好的語言，但可能會出錯。

XML API 多部分上傳作業

本頁面說明 Cloud Storage 中的 XML API 分塊上傳。這種上傳方法會分批上傳檔案，然後使用最終要求將檔案組合成單一物件。XML API 分塊上傳與 Amazon S3 分塊上傳相容。

總覽

XML API 多部分上傳可讓您分批上傳資料，然後組合成最終物件。這種行為有幾項優點，尤其是對大型檔案而言：

您可以同時上傳多個部分，縮短上傳完整資料所需的時間。
如果其中一項上傳作業失敗，您只需要重新上傳整體物件的一部分，不必從頭開始。
由於未事先指定檔案總大小，您可以使用 XML API 分段上傳功能進行串流上傳，或在上傳時即時壓縮資料。

XML API 多部分上傳作業有三個必要步驟：

使用 POST 要求啟動上傳，包括指定完成物件應具備的任何中繼資料。回應會傳回 UploadId，您可以在與上傳相關的所有後續要求中使用。
使用一或多個 PUT 要求上傳資料。
使用 POST 要求完成上傳。這項要求會覆寫值區中任何同名的現有物件。

多部分上傳作業及其上傳的部分，在 bucket 中維持未完成或閒置狀態的時間長度沒有限制。

成功上傳的部分會計入每月儲存空間用量。
您可以使用物件生命週期管理功能，在多部分上傳作業達到指定時間長度時自動移除，避免累積過多未完成的多部分上傳作業。

注意事項

使用 XML API 分塊上傳時，須遵守下列限制：

分塊大小有限制，包括大小下限、大小上限，以及用於組合完整上傳內容的分塊數量。
要求不支援預先條件。
使用這個方法上傳的物件沒有 MD5 雜湊。
Google Cloud 控制台或 Google Cloud CLI 不支援這種上傳方法。

使用 XML API 分塊上傳時，請注意下列事項：

XML API 分塊上傳作業有特定 IAM 權限。如果您使用自訂 IAM 角色，請確認這些角色具備所需權限。
雖然您可以啟動上傳作業並上傳部分內容，但如果完成上傳的要求會覆寫具有保留或未完成保留期限的物件，則該要求會失敗。
您可以列出 bucket 中進行中的上傳作業，但只有已完成的上傳作業會顯示在 bucket 的一般物件清單中。
如果上傳的部分從未使用，可能需要支付提前刪除費用。

用戶端程式庫如何使用 XML API 多部分上傳

本節說明如何使用支援 XML API 分段上傳功能的用戶端程式庫，執行 XML API 分段上傳。

用戶端程式庫

Java

詳情請參閱 Cloud Storage Java API 參考文件。

如要向 Cloud Storage 進行驗證，請設定應用程式預設憑證。詳情請參閱「設定用戶端程式庫的驗證機制」。

以下範例會啟動多部分上傳：


import com.google.cloud.storage.HttpStorageOptions;
import com.google.cloud.storage.MultipartUploadClient;
import com.google.cloud.storage.MultipartUploadSettings;
import com.google.cloud.storage.multipartupload.model.CreateMultipartUploadRequest;
import com.google.cloud.storage.multipartupload.model.CreateMultipartUploadResponse;
import java.util.HashMap;
import java.util.Map;

public class CreateMultipartUpload {
  public static void createMultipartUpload(String projectId, String bucketName, String objectName) {
    // The ID of your GCP project
    // String projectId = "your-project-id";

    // The ID of your GCS bucket
    // String sourceBucketName = "your-unique-bucket-name";

    // The ID of your GCS object
    // String sourceObjectName = "your-object-name";

    HttpStorageOptions storageOptions =
        HttpStorageOptions.newBuilder().setProjectId(projectId).build();
    MultipartUploadSettings mpuSettings = MultipartUploadSettings.of(storageOptions);
    MultipartUploadClient mpuClient = MultipartUploadClient.create(mpuSettings);

    System.out.println("Initiating multipart upload for " + objectName);

    Map<String, String> metadata = new HashMap<>();
    metadata.put("key1", "value1");
    String contentType = "text/plain";
    CreateMultipartUploadRequest createRequest =
        CreateMultipartUploadRequest.builder()
            .bucket(bucketName)
            .key(objectName)
            .metadata(metadata)
            .contentType(contentType)
            .build();

    CreateMultipartUploadResponse createResponse = mpuClient.createMultipartUpload(createRequest);
    String uploadId = createResponse.uploadId();
    System.out.println("Upload ID: " + uploadId);
  }
}

下列範例會上傳個別物件部分：


import com.google.cloud.storage.HttpStorageOptions;
import com.google.cloud.storage.MultipartUploadClient;
import com.google.cloud.storage.MultipartUploadSettings;
import com.google.cloud.storage.RequestBody;
import com.google.cloud.storage.multipartupload.model.UploadPartRequest;
import com.google.cloud.storage.multipartupload.model.UploadPartResponse;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.Random;

public class UploadPart {
  public static void uploadPart(
      String projectId, String bucketName, String objectName, String uploadId, int partNumber)
      throws IOException {
    // The ID of your GCP project
    // String projectId = "your-project-id";

    // The ID of your GCS bucket
    // String bucketName = "your-unique-bucket-name";

    // The ID of your GCS object
    // String objectName = "your-object-name";

    // The ID of the multipart upload
    // String uploadId = "your-upload-id";

    // The part number of the part being uploaded
    // int partNumber = 1;

    HttpStorageOptions storageOptions =
        HttpStorageOptions.newBuilder().setProjectId(projectId).build();
    MultipartUploadSettings mpuSettings = MultipartUploadSettings.of(storageOptions);
    MultipartUploadClient mpuClient = MultipartUploadClient.create(mpuSettings);

    // The minimum part size for a multipart upload is 5 MiB, except for the last part.
    byte[] bytes = new byte[5 * 1024 * 1024];
    new Random().nextBytes(bytes);
    RequestBody requestBody = RequestBody.of(ByteBuffer.wrap(bytes));

    System.out.println("Uploading part " + partNumber);
    UploadPartRequest uploadPartRequest =
        UploadPartRequest.builder()
            .bucket(bucketName)
            .key(objectName)
            .partNumber(partNumber)
            .uploadId(uploadId)
            .build();

    UploadPartResponse uploadPartResponse = mpuClient.uploadPart(uploadPartRequest, requestBody);

    System.out.println("Part " + partNumber + " uploaded with ETag: " + uploadPartResponse.eTag());
  }
}

下列範例會列出物件部分：


import com.google.cloud.storage.HttpStorageOptions;
import com.google.cloud.storage.MultipartUploadClient;
import com.google.cloud.storage.MultipartUploadSettings;
import com.google.cloud.storage.multipartupload.model.ListPartsRequest;
import com.google.cloud.storage.multipartupload.model.ListPartsResponse;
import com.google.cloud.storage.multipartupload.model.Part;

public class ListParts {
  public static void listParts(
      String projectId, String bucketName, String objectName, String uploadId) {
    // The ID of your GCP project
    // String projectId = "your-project-id";

    // The ID of your GCS bucket
    // String bucketName = "your-unique-bucket-name";

    // The ID of your GCS object
    // String objectName = "your-object-name";

    // The ID of the multipart upload
    // String uploadId = "your-upload-id";

    HttpStorageOptions storageOptions =
        HttpStorageOptions.newBuilder().setProjectId(projectId).build();
    MultipartUploadSettings mpuSettings = MultipartUploadSettings.of(storageOptions);
    MultipartUploadClient mpuClient = MultipartUploadClient.create(mpuSettings);

    System.out.println("Listing parts for upload ID: " + uploadId);

    ListPartsRequest listPartsRequest =
        ListPartsRequest.builder().bucket(bucketName).key(objectName).uploadId(uploadId).build();

    ListPartsResponse listPartsResponse = mpuClient.listParts(listPartsRequest);

    if (listPartsResponse.parts() == null || listPartsResponse.parts().isEmpty()) {
      System.out.println("No parts have been uploaded yet.");
      return;
    }

    System.out.println("Uploaded Parts:");
    for (Part part : listPartsResponse.parts()) {
      System.out.println("  - Part Number: " + part.partNumber());
      System.out.println("    ETag: " + part.eTag());
      System.out.println("    Size: " + part.size() + " bytes");
      System.out.println("    Last Modified: " + part.lastModified());
    }
  }
}

以下範例會完成多部分上傳：


import com.google.cloud.storage.HttpStorageOptions;
import com.google.cloud.storage.MultipartUploadClient;
import com.google.cloud.storage.MultipartUploadSettings;
import com.google.cloud.storage.multipartupload.model.CompleteMultipartUploadRequest;
import com.google.cloud.storage.multipartupload.model.CompleteMultipartUploadResponse;
import com.google.cloud.storage.multipartupload.model.CompletedMultipartUpload;
import com.google.cloud.storage.multipartupload.model.CompletedPart;
import java.util.List;

public class CompleteMultipartUpload {
  public static void completeMultipartUpload(
      String projectId,
      String bucketName,
      String objectName,
      String uploadId,
      List<CompletedPart> completedParts) {

    // The ID of your GCP project
    // String projectId = "your-project-id";

    // The ID of your GCS bucket
    // String bucketName = "your-unique-bucket-name";

    // The ID of your GCS object
    // String objectName = "your-object-name";

    // The ID of the multipart upload
    // String uploadId = "your-upload-id";

    // The list of completed parts from the UploadPart responses.
    // List<CompletedPart> completedParts = ...;

    HttpStorageOptions storageOptions =
        HttpStorageOptions.newBuilder().setProjectId(projectId).build();
    MultipartUploadSettings mpuSettings = MultipartUploadSettings.of(storageOptions);
    MultipartUploadClient mpuClient = MultipartUploadClient.create(mpuSettings);

    System.out.println("Completing multipart upload for " + objectName);

    CompletedMultipartUpload completedMultipartUpload =
        CompletedMultipartUpload.builder().parts(completedParts).build();

    CompleteMultipartUploadRequest completeRequest =
        CompleteMultipartUploadRequest.builder()
            .bucket(bucketName)
            .key(objectName)
            .uploadId(uploadId)
            .multipartUpload(completedMultipartUpload)
            .build();

    CompleteMultipartUploadResponse completeResponse =
        mpuClient.completeMultipartUpload(completeRequest);

    System.out.println(
        "Upload complete for "
            + completeResponse.key()
            + " in bucket "
            + completeResponse.bucket());
    System.out.println("Final ETag: " + completeResponse.etag());
  }
}

Node.js

詳情請參閱 Cloud Storage Node.js API 參考文件。

如要向 Cloud Storage 進行驗證，請設定應用程式預設憑證。詳情請參閱「設定用戶端程式庫的驗證機制」。

您可以使用 uploadFileInChunks 方法，執行 XML API 分塊上傳。例如：

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// The ID of your GCS bucket
// const bucketName = 'your-unique-bucket-name';

// The path of file to upload
// const filePath = 'path/to/your/file';

// The size of each chunk to be uploaded
// const chunkSize = 32 * 1024 * 1024;

// Imports the Google Cloud client library
const {Storage, TransferManager} = require('@google-cloud/storage');

// Creates a client
const storage = new Storage();

// Creates a transfer manager client
const transferManager = new TransferManager(storage.bucket(bucketName));

async function uploadFileInChunksWithTransferManager() {
  // Uploads the files
  await transferManager.uploadFileInChunks(filePath, {
    chunkSizeBytes: chunkSize,
  });

  console.log(`${filePath} uploaded to ${bucketName}.`);
}

uploadFileInChunksWithTransferManager().catch(console.error);

Python

詳情請參閱 Cloud Storage Python API 參考文件。

如要向 Cloud Storage 進行驗證，請設定應用程式預設憑證。詳情請參閱「設定用戶端程式庫的驗證機制」。

您可以使用 upload_chunks_concurrently 方法，執行 XML API 分塊上傳。例如：

def upload_chunks_concurrently(
    bucket_name,
    source_filename,
    destination_blob_name,
    chunk_size=32 * 1024 * 1024,
    workers=8,
):
    """Upload a single file, in chunks, concurrently in a process pool."""
    # The ID of your GCS bucket
    # bucket_name = "your-bucket-name"

    # The path to your file to upload
    # source_filename = "local/path/to/file"

    # The ID of your GCS object
    # destination_blob_name = "storage-object-name"

    # The size of each chunk. The performance impact of this value depends on
    # the use case. The remote service has a minimum of 5 MiB and a maximum of
    # 5 GiB.
    # chunk_size = 32 * 1024 * 1024 (32 MiB)

    # The maximum number of processes to use for the operation. The performance
    # impact of this value depends on the use case. Each additional process
    # occupies some CPU and memory resources until finished. Threads can be used
    # instead of processes by passing `worker_type=transfer_manager.THREAD`.
    # workers=8

    from google.cloud.storage import Client, transfer_manager

    storage_client = Client()
    bucket = storage_client.bucket(bucket_name)
    blob = bucket.blob(destination_blob_name)

    transfer_manager.upload_chunks_concurrently(
        source_filename, blob, chunk_size=chunk_size, max_workers=workers
    )

    print(f"File {source_filename} uploaded to {destination_blob_name}.")


if __name__ == "__main__":
    argparse = argparse.ArgumentParser(
        description="Upload a file to GCS in chunks concurrently."
    )
    argparse.add_argument(
        "--bucket_name", help="The name of the GCS bucket to upload to."
    )
    argparse.add_argument(
        "--source_filename", help="The local path to the file to upload."
    )
    argparse.add_argument(
        "--destination_blob_name", help="The name of the object in GCS."
    )
    argparse.add_argument(
        "--chunk_size",
        type=int,
        default=32 * 1024 * 1024,
        help="The size of each chunk in bytes (default: 32 MiB). The remote\
              service has a minimum of 5 MiB and a maximum of 5 GiB",
    )
    argparse.add_argument(
        "--workers",
        type=int,
        default=8,
        help="The number of worker processes to use (default: 8).",
    )
    args = argparse.parse_args()
    upload_chunks_concurrently(
        args.bucket_name,
        args.source_filename,
        args.destination_blob_name,
        args.chunk_size,
        args.workers,
    )

後續步驟

探索 Cloud Storage 的其他上傳方法。
瞭解截斷指數輪詢和重試要求的時機。

XML API 多部分上傳作業 透過集合功能整理內容 你可以依據偏好儲存及分類內容。

總覽

注意事項

用戶端程式庫如何使用 XML API 多部分上傳

用戶端程式庫

Java

Node.js

Python

後續步驟

XML API 多部分上傳作業