To successfully relocate buckets, define your objectives and understand your bucket's usage before initiating a bucket relocation. The following sections describe the key planning steps.
Determine your bucket relocation type
When relocating your bucket, it's important to understand that there might be a write downtime period during the final synchronization step where you can't update or upload new objects. Additionally, you won't be able to change your bucket's configuration during the relocation process. To determine if your relocation involves downtime, see Relocation types.
Review the unsupported features and the compatibility requirements
Identify any configurations in your source bucket that don't support bucket relocation and the configurations that require action to support bucket relocation. If your bucket uses unsupported configurations that can't be modified, or if the source or destination is an unsupported location, you must manually copy the objects into a different bucket in the destination location, instead of relocating the bucket with its objects. For details, see Move data between buckets.
The following sections describe the unsupported features and the compatibility requirements.
Unsupported features
The following table describes the features that aren't compatible with bucket relocation. In some cases, you can reconfigure a feature to support bucket relocation:
| Feature | Compatibility status | Required action before initiating the bucket relocation |
|---|---|---|
| Hierarchical namespace | Not supported for bucket relocations with write downtime. | If a bucket has hierarchical namespace enabled, you can only relocate it if the process doesn't involve write downtime |
| Appspot buckets | Not supported. | You can't relocate Appspot buckets. Consider migrating Container Registry to Artifact Registry as a workaround for default buckets created by App Engine. |
| Firebase buckets | Not supported. | You can't relocate Firebase buckets. |
| Object holds | Not supported. You can't relocate buckets containing objects with holds. |
To use bucket relocation, remove the object holds. |
| Managed folders | Not supported. You can't relocate buckets containing managed folders. |
To use bucket relocation, delete the managed folders. |
| Customer-managed encryption keys (CMEK) or customer-supplied encryption keys (CSEK) | Not supported for relocations with write downtime. | To use bucket relocation, remove the customer-managed encryption keys or customer-supplied encryption keys. After removal, Cloud Storage automatically protects your data using the standard Cloud Storage encryption. |
| Anywhere Cache | Supported for bucket relocations without write downtime and partially supported for bucket relocations with write downtime. | For relocating buckets with write downtime, disable Anywhere Cache before the final synchronization step. |
| Bucket lock | Not supported when retention policies are locked. | Unlock the retention policies. |
| Tags | Not supported for relocations with write downtime. |
You must detach tags that are directly attached to the bucket. If any of the tags being detached from your source bucket are used for access control, you must use an alternative method to set up the IAM roles to secure the data in your bucket. To do so, complete the following steps:
|
| Inventory report configurations | Existing inventory report configurations aren't preserved during the relocation process. | Save your existing inventory report configurations manually before starting the relocation process, so that you can recreate them after the relocation process is complete. For information about managing inventory report configurations, see Create and manage inventory report configurations. |
Feature compatibility during bucket relocation
The following table describes how other Cloud Storage capabilities work when you relocate a bucket. The behavior might vary depending on the relocation mode:
| Feature | Relocation with write downtime | Relocation without write downtime |
|---|---|---|
| Autoclass behavior | Autoclass is temporarily paused during the final synchronization step. The pause might delay objects moving to colder storage classes. For details, see Autoclass object transitions when relocating buckets. | Autoclass behavior isn't affected. |
| BigQuery and BigLake tables | BigLake external tables and BigQuery tables using Apache Iceberg become inaccessible after a relocation and require manual recreation. Automatic detection of impacted tables isn't available. | Supported. |
| Object size limit | 2 TB limit applies to object sizes. | No size limit. |
| Multipart uploads |
Compatibility and behavior for multipart uploads depend on the status of the upload when you start a bucket relocation:
|
Compatibility and behavior for multipart uploads depend on the status of the upload when you start a bucket relocation:
|
| Resumable uploads | Not supported. In-progress resumable uploads must be finalized before the final synchronization step of the bucket relocation process to avoid data loss. |
Supported. |
| Relocation across projects | Not supported. You can't relocate buckets across projects. |
Supported. |
| Metadata updates | Not supported. You can't update a bucket's metadata during relocation. |
Supported. |
| Request rate ramp-up | Relocated buckets are subject to the same request rate ramp-up guidelines as newly created buckets. | Not applicable. |
Analyze the bucket characteristics
To estimate your bucket relocation time, analyze your bucket's characteristics and usage, considering the following factors:
At-rest bytes: The total amount of data stored in the bucket impacts storage costs and transfer time.
Replication: Replicating the bucket to other regions, either synchronously or asynchronously, impacts data availability, durability, and cost. For details, see Data availability and durability.
Data transfer: The amount of data transferred out of the bucket during the relocation impacts the data transfer cost calculations. To calculate your bucket's data transfer costs, see Cloud Storage pricing.
Usage patterns: Understanding bucket activity levels, or how busy the bucket is, through usage patterns helps you prevent unexpected conflicts during the relocation. To understand your bucket's usage patterns, you can analyze your logs. For details, see Usage logs and storage logs.
Bucket write operations: Frequent bucket write operations during the relocation process increase the cost and duration. To understand how frequently objects are being written to your bucket, see Overview of monitoring in Cloud Storage.
Define your relocation goals
Based on your analysis of the bucket characteristics, identify the reasons for moving your bucket. The following are common goals for relocating a bucket:
Cost management: Reduce storage costs by moving to a lower-cost region or minimize data transfer costs by moving data closer to its access location. You'll need to calculate your Cloud Storage and data transfer costs and compare them to potential costs in different locations. For details about calculating costs for your Cloud Storage, see Cloud Storage pricing.
Performance enhancement: Improve data access speed and application performance by relocating the bucket closer to users or applications. To do so, identify the geographic regions where performance is critical and relocate your buckets.
Reliability improvement: Enhance data durability and disaster recovery capabilities by using dual- or multi-region configurations.
Decide on the bucket location
Based on your analysis and goals, choose the most suitable storage location for the bucket you're relocating from the following options:
Single-region: Store data in a single region that is cost-effective for applications with users concentrated in one geographic area.
Dual-region: Maintain two copies of your data in two regions within the same continent, providing higher availability and disaster recovery capabilities within a specific geographic area.
Multi-region: Distribute data across multiple regions, offering the highest level of availability and durability.
To learn more about choosing a location, see Considerations for choosing a location.
Understand the factors that affect the relocation time
Several factors affect the relocation time, and understanding them can help to estimate the required time. While these factors offer a helpful starting point for planning and scheduling your relocation, actual relocation times might be longer or shorter than the estimated time. Therefore, when scheduling your relocation, add buffer time to account for potential delays. The following sections describe the factors that affect the relocation time.
Relocation service limits
The following table describes the limits that affect the relocation time:
| Factor | Value | Description |
|---|---|---|
| Maximum request rate per job | 10,000 objects per second |
These are the number of copy requests the service can handle per second.
A higher request rate means more files can be moved simultaneously. If your bucket has many small files, a high request rate speeds up the migration. If you have only a few large files, this factor has less impact. |
| Maximum overall bandwidth per project | 10 GBps |
This is the maximum speed or bandwidth at which you can transfer data for
a single project within a source location. If you're moving multiple buckets
within the same project, then the buckets share the bandwidth.
A higher bandwidth means more data can be transferred at once. Even with a high request rate, if the bandwidth is small, the overall transfer is slow. |
| Maximum bandwidth per single object | 8 MBps |
This is the maximum speed at which you can transfer a single object.
A higher bandwidth per single object means you can transfer the objects at a faster rate. This is the speed limit for moving one object at a time. Even with a high request rate and high bandwidth per bucket, if individual objects have a speed limit, they can take more time to transfer. |
| Maximum concurrent relocations per project | 30 relocations | The bucket relocation service supports up to 30 concurrent relocations from the same location within a project. |
Relocation Time to Live limit
To help with resource utilization and prevent relocations from running indefinitely, a Time to Live (TTL) limit applies to all bucket relocations. TTL refers to the maximum time allowed for the entire relocation process to complete.
The maximum time allowed to complete a bucket relocation is 28 days and includes all phases of the relocation process, such as the initial copy, incremental updates, and final synchronization.
If the relocation process exceeds the 28-day TTL limit, the relocation operation fails.
Ongoing bucket activity
If you continue to write new objects, delete existing ones, or update objects in the bucket during the relocation, these operations compete for resources with the copy requests and can slow down the relocation process.
Lifecycle rules
If you have lifecycle rules configured for your bucket, such as automatically deleting or archiving objects after a specific time, these actions increase the overall relocation time.
Configure Storage Intelligence
You must configure Storage Intelligence for both the source and destination locations. You can configure Storage Intelligence at different levels of your Google Cloud resource hierarchy. You can also use inclusion and exclusion filters to include relevant buckets in your Storage Intelligence configuration. For details, see Configure Storage Intelligence.
Enable soft delete
Bucket relocation requires that you enable soft delete on the bucket and set the retention duration to at least seven days. The retention duration is the amount of time that soft delete keeps deleted objects before permanently deleting them. For information about how to configure the soft delete retention duration, see Use soft delete.
Check quotas and limits
Quotas and cloud capacity assessments are tied to specific regions or zones. As a result, when you move a bucket to a new location, you need to verify that the new location has sufficient quotas to accommodate the bucket's data. For more information about quotas and limits, see Quotas and limits.
What's next
- Learn how to relocate buckets.