About object deletion

Deleting objects that you no longer need helps you reduce storage costs, comply with data retention requirements, and keep your buckets organized. Cloud Storage supports two primary methods for deleting objects:

  • Single-object deletion: deletes individual objects for targeted updates.
  • Bulk object deletion: deletes multiple objects simultaneously or on an automated schedule.

For each deletion method, Cloud Storage provides various tools to delete objects manually, programmatically, or through automated policies. Review these deletion methods and available tools to choose the best approach for your workload.

For step-by-step instructions to delete objects, see Delete objects.

Single-object deletion

Single-object deletion provides precise control over specific data. You typically delete single objects when you need to perform immediate, targeted updates to your bucket's data. Common use cases for single-object deletion include the following:

  • Correcting errors: removing an object that you uploaded accidentally or an object that contains incorrect data.

  • Cleanup: manually removing temporary objects or logs after completing a task.

  • Application logic: automatically removing an object when an application event occurs, such as a user deleting a profile photo.

  • Security and compliance: deleting objects to remove sensitive information.

Tools for single-object deletion

To delete a single object from your bucket, use any of the following tools. Each tool sends an individual DELETE request per object.

Tool Description
Google Cloud console Deletes an individual object in the Google Cloud console.
Google Cloud CLI Deletes an object by using the gcloud storage rm command.
Client libraries Deletes an object programmatically by using a supported language, such as C++, C#, Go, Java, Node.js, PHP, Python, or Ruby.
REST APIs Deletes an object by using the JSON API or XML API.

Bulk object deletion

Bulk object deletion is designed for high-efficiency operations on large datasets. Use bulk deletion to manage storage costs and automate data hygiene across buckets. Common use cases for bulk object deletion include the following:

  • Cost optimization: removing large datasets that you no longer need for operations, such as old logs or temporary build artifacts.

  • Compliance automation: automatically enforcing data retention policies by deleting objects after a specific retention period, such as 30 days.

  • Bucket migration or decommissioning: clearing millions or billions of objects when migrating data or closing a project.

  • Data pipeline cleanup: removing intermediate processing objects after a pipeline generates a final report or dataset.

Tools for bulk object deletion

To delete objects in bulk, use any of the following tools. These tools are optimized for higher throughput and reduced overhead compared to individual requests.

Tool Description
Google Cloud console Deletes up to one million objects interactively. After you initiate a deletion request, Cloud Storage processes the deletion in the background. You can check the status of your bulk deletion by clicking the Notifications button () in the Google Cloud console header.
Google Cloud CLI Deletes all objects that share a common prefix by using the gcloud storage rm --recursive command. This command is suitable for small-to-medium-scale bulk deletions from the command line.
Multi-object delete XML API

Deletes up to 1,000 objects programmatically in a single POST request. The multi-object delete XML API is interoperable with the Amazon S3 API. As a result, you can use your existing Amazon S3 tools, libraries, and workflows to manage objects in Cloud Storage. The XML API is useful if you are migrating data or managing resources across both platforms.

When you use the XML API to delete multiple objects, Cloud Storage generates data access audit logs. If you enable data access audit logs, these logs include details about the overall request and individual object deletions. For details, see Audit logs for multi-object delete XML API.

The multi-object delete XML API has the following limitations:

  • Governance mode bypass: Unlike Amazon S3, Cloud Storage doesn't allow bypassing governance mode during deletion. In Amazon S3, you can delete objects locked in governance mode by specifying the x-amz-bypass-governance-retention header. Cloud Storage strictly enforces object retention policies to maintain data immutability.
  • Delete markers: Cloud Storage doesn't support Amazon S3 delete markers for single-object or multi-object deletions. To restore a noncurrent object in Cloud Storage, you must copy the specific version to the live object by using an XML API PUT request.
  • Conditional delete operations: The Cloud Storage multi-object delete XML API doesn't support conditional deletions based on ETag, Last-Modified, or Size. Requests that include these parameters return an HTTP 400 Bad Request status code. To perform conditional operations in Cloud Storage, use headers such as x-goog-if-generation-match or x-goog-if-metageneration-match.
Storage batch operations

Deletes up to billions of objects based on a manifest file by using a fully managed service with built-in tracking and retries.

Use storage batch operations to perform the following tasks:

  • Delete millions or billions of objects.
  • Enhance reliability with built-in tracking, retries, and error handling.
  • Reduce client-side overhead compared to scripting individual API calls.
Object Lifecycle Management Deletes objects automatically based on user-defined lifecycle rules, such as object age or storage class. To delete objects in bulk, configure an Object Lifecycle Management rule on your bucket. Set the condition Age to 0 days and the action to Delete. After you set the rule, Cloud Storage performs the bulk deletion asynchronously.
Batch API requests Batches multiple deletion requests into a single HTTP request when you use supported client libraries or the JSON API directly. This batching reduces client-side connection overhead and latency.

Choose a deletion method

The optimal method and tool for object deletion depend on the object count and your workflow requirements. Use the following decision matrix to select an approach for your use case:

Use case Deletion method Recommended tool
Delete an individual object or a small number of specific objects Single-object deletion Google Cloud console, gcloud storage rm, client libraries, or REST APIs
Delete up to 1,000 objects in a single programmatic request Bulk object deletion Multi-object delete XML API, Amazon S3 CLI, or Amazon S3-compatible client libraries (such as Boto3)
Delete objects that share a common prefix Bulk object deletion gcloud storage rm --recursive
Delete up to one million objects interactively in the browser Bulk object deletion Google Cloud console
Delete millions or billions of objects asynchronously with built-in tracking Bulk object deletion Storage batch operations
Delete objects automatically based on conditions (such as age or storage class) Bulk object deletion Object Lifecycle Management

What's next