Sources and sinks

Storage Transfer Service supports transfers across cloud object stores, URL lists, and on-premises or cloud file systems. The following table details the supported sources, destinations, and transfer types.

Storage system Source Destination Transfer type Notes
Cloud Storage Agentless or agent-based

Cloud Storage is a valid sink for transfers from all sources.

Transfers from Amazon S3, Azure Blob Storage, URL lists, and between Cloud Storage buckets are agentless. Transfers paired with file systems, HDFS, or S3-compatible storage are agent-based.

To configure Cloud Storage as a source or sink, refer to the following pages:

Amazon Simple Storage Service (Amazon S3) Agentless

Transfers to Cloud Storage only.

To configure Amazon S3, refer to Configure access to a source: Amazon S3.

S3-compatible storage Agent-based

Transfers to Cloud Storage only.

To configure S3-compatible storage, refer to Configure access to a source: S3-compatible storage.

Azure Blob Storage, including Azure Data Lake Storage Gen2 Agentless

Transfers to Cloud Storage only.

To configure Azure storage options, refer to Configure access to a source: Microsoft Azure Storage.

Publicly accessible or signed HTTP and HTTPS URLs Agentless

Storage Transfer Service accepts a TSV file specifying a list of URLs to transfer.

Transfers to Cloud Storage only.

To configure access to a URL list, refer to Configure access to a source: URL list. For formatting details, see Create a URL list.

File system Agent-based

Storage Transfer Service supports transfers:

  • From file systems to Cloud Storage
  • From Cloud Storage to file systems
  • Between two file systems

See Manage agent pools to configure agents and agent pools.

To configure a file system as a source or sink, refer to the following pages:

Hadoop Distributed File System, HDFS Agent-based

Transfers to Cloud Storage only.

To configure a transfer from an HDFS source, refer to Transfer from HDFS to Cloud Storage.