High-level overview

This page describes the high-level procedure for ordering, installing, and returning Transfer Appliance. It includes the expected interactions between you and Google.

Before you begin

Before ordering, use the following checklist to ensure that your environment supports using Transfer Appliance.

  • Review the Transfer Appliance specifications and confirm the following for your facility:

    • Power capabilities
    • Cooling capabilities
    • Physical space requirements
  • Identify the servers and the folders that contain data to transfer.

  • Confirm that employees performing the transfer have the correct permissions to access the data for the transfer.

  • Review that your data meets the following criteria:

    • Individual files are less than 5 terabytes (TB) in size.

    • File names follow object naming guidelines.

    • Files are regular files or Unix-style hidden files. Unix-style hidden files start with a . character.

    • Folders contain files. Empty folders are not created in Cloud Storage, because objects don't reside within subdirectories within a Cloud Storage bucket.

    • The maximum path length is 1024 bytes, which includes the optional object prefix specified when you provide Transfer Appliance Team with bucket configuration details.

Prepare for the appliance

To prepare for Transfer Appliance, do the following:

  • When you're ready to transfer your data, apply for Transfer Appliance from the same Google Cloud project as your destination Cloud Storage bucket. You must have the required permissions on the project to successfully apply for the Transfer Appliance.

  • When applying for Transfer Appliance, you can choose to use your appliance in either offline mode or online mode.

    • Offline mode allows you to copy data to the appliance. When the appliance is full, you send the appliance back to us.

    • Online mode enables streaming data copied to the appliance to your Cloud Storage bucket.

  • If you select online mode, you can toggle between offline and online mode once you have the appliance. If you select offline mode, you cannot toggle modes once you receive your appliance.

Project Setup and Key Download

  1. Ensure you have the following permissions:

    • iam.serviceAccounts.getIamPolicy
    • resourcemanager.projects.getIamPolicy
    • resourcemanager.projects.setIamPolicy
    • storage.buckets.getIamPolicy
    • storage.buckets.setIamPolicy

    You will be able to get these required permissions through custom roles or predefined roles. You can search for predefined roles for a given permission.

  2. Complete the project setup:

    1. Go to appliances page
    2. Find and select the appliance
    3. Click on the Open Setup Checklist button
    4. Complete the following steps:
      1. Enable API: enables the APIs required by the appliance in the project that you ordered the appliance from. For online transfer, this enables the Transfer Appliance API, the Pub/Sub API, and the Storage Transfer Service API.
      2. Authorize the service accounts: creates the service accounts used for transfer and grants them access to your project and to the destination bucket. For online transfer, do the following:
        1. Click Create service account to create the appliance service account. The appliance service account has the format ID@PROJECT_ID.iam.gserviceaccount.com. The appliance uses this service account, through the key that you upload to the appliance, to create and run Storage Transfer Service jobs and to upload your data to the destination bucket.
        2. Grant the listed roles to the appliance service account and to the Storage Transfer Service service agent of your project (project-PROJECT_NUMBER@storage-transfer-service.iam.gserviceaccount.com).
      3. (only for online transfer) Create and Download a key for the appliance service account that you created in the previous step. The key must belong to the appliance service account. Don't create a key for any of the other service accounts listed in the checklist.

    If you can't use the setup checklist, for example because your organization doesn't allow service account keys to be created in the order project, you can configure the APIs, service accounts, and roles manually, including using a service account that belongs to a different project. For the required APIs and the least-privilege roles, see Prepare for online transfer.

Receive the appliance

To receive an appliance, follow these steps:

  1. Track the appliance shipped to you. We'll email you when it is delivered to your shipping address.

    You can also track your shipment directly: Track shipment

    After you receive the appliance, confirm the Appliance Receipt. Then, verify the package contents to ensure that you have everything required to use the appliance.

  2. Before connecting the appliance to your network, validate that it hasn't been tampered with during shipping and submit the Transfer Appliance Attestation Passcode returned by the appliance in the appliance detail page of the Google Cloud console.

    If the device was not tampered with during shipping, the credentials will be displayed in the Google Cloud console. Otherwise, the Google Cloud console will show an error message. If you encounter this error, contact the Transfer Appliance Team at data-support@google.com for assistance.

  3. After receiving login credentials from Google Cloud console, do the following:

    1. Configure IP network ports.
    2. Connect the appliance to your network.
    3. Configure the appliance software.

Copying data to the appliance

To capture your data, follow these steps:

  1. Use copy tools of your choice to copy data to the appliance.

    • If you selected an online-capable appliance and enabled online copy on the appliance: The appliance streams the data you copy to the appliance to your Cloud Storage bucket. Files and empty directories are deleted from the appliance after they're successfully copied to your Cloud Storage bucket.

    • If you selected an offline-only device or disabled online copy on the appliance: After you complete all copying to the appliance, finalize the copied data to prepare for shipping.

    • All files copied to the appliance must comply with Cloud Storage file size limits and naming guidelines.

  2. Verify that you copied all data to the appliance before finalizing.

  3. Finalize copied data.

  4. Enter the passcode returned by the ta finalize command in the form included in the email titled Google Transfer Appliance Return Instructions sent by the Transfer Appliance Team.

    To view the passcode again, run ta status.

    The Transfer Appliance Team responds with the shipping label for the Transfer Appliance for the given passcode.

  5. Once you receive the shipping label, turn off the appliance and disconnect all cables to it.

  6. Once you have the shipping label, package and return the appliance.

Complete your copy

After returning the appliance, do the following to complete your transfer:

  1. Verify that we received your appliance.

    Transfer Appliance Team emails you when we receive your appliance at our data transfer center. We'll start transferring data from the appliance to the destination Cloud Storage bucket indicated in the appliance order form.

  2. Verify that we completed your transfer.

    Transfer Appliance Team emails you when we've completed transferring data from the appliance to your Cloud Storage bucket.

  3. Validate your data in the destination Cloud Storage bucket to verify that your data transferred as you expected. We recommend that you don't delete your source data until you verify that it successfully transferred to your Cloud Storage bucket.

    • Data that's been transferred from your appliance is displayed in your Cloud Storage bucket.

    • The following additional directories are created:

      • A storage-transfer directory that contains your Storage Transfer Service logs.
      • A transferappliance directory that logs successful and failed transfers from the appliance.
      • Sometimes, a cloud-ingest directory that is used to coordinate the data uploaded from the appliance to your destination Transfer Appliance bucket.

    You can delete these directories once your data transfer is complete and the data is fully available in your Transfer Appliance bucket.

  4. Optional: You may request a Transfer Appliance wipe certificate to verify that we've securely wiped all data after copying it to Cloud Storage. If you're interested in a wipe certificate, contact Transfer Appliance Team.