Sources and sinks

Storage Transfer Service supports transfers across cloud object stores, URL lists, and on-premises or cloud file systems. The following table details the supported sources, destinations, and transfer types.

Storage system Source Destination Transfer type Notes
Cloud Storage Agentless or agent-based

Cloud Storage is a valid sink for transfers from all sources.

Transfers between cloud systems are agentless. Transfers paired with file systems are agent-based.

To configure Cloud Storage as a source or sink, refer to the following pages:

Amazon Simple Storage Service (Amazon S3) Agentless

Transfers to Cloud Storage only.

To configure Amazon S3, refer to Configure access to a source: Amazon S3.

S3-compatible storage Agent-based

Transfers to Cloud Storage only.

To configure S3-compatible storage, refer to Configure access to a source: S3-compatible storage.

Azure Blob Storage, including Azure Data Lake Storage Gen2 Agentless

Transfers to Cloud Storage only.

To configure Azure storage options, refer to Configure access to a source: Microsoft Azure Storage.

Publicly accessible or signed HTTP and HTTPS URLs Agentless

Storage Transfer Service accepts a TSV file specifying a list of URLs to transfer.

Transfers to Cloud Storage only.

To configure access to a URL list, refer to Configure access to a source: URL list. For formatting details, see Create a URL list.

File system Agent-based

Storage Transfer Service supports transfers:

  • From file systems to Cloud Storage
  • From Cloud Storage to file systems
  • Between two file systems

See Manage agent pools to configure agents and agent pools.

To configure a file system as a source or sink, refer to the following pages:

Hadoop Distributed File System, HDFS Agent-based

Transfers to Cloud Storage only.

To configure a transfer from an HDFS source, refer to Transfer from HDFS to Cloud Storage.