Configure a source PostgreSQL database

You can stream ongoing changes from a source PostgreSQL database to a supported destination, such as BigQuery or Cloud Storage, using Datastream. To capture database activity, Datastream uses logical decoding to read changes from write-ahead log (WAL) files through a publication and replication slot. For more information, see Work with PostgreSQL database WAL files.

Publication and replication slots

When you configure your PostgreSQL source, you need to create a publication and a replication slot.

A publication is a set of changes generated from a table, or a group of tables, that you want to replicate using logical replication.

A replication slot is a PostgreSQL feature which ensures that a stream of changes stored in a WAL log file is replicated to the destination in the correct order.

Each replication slot streams a sequence of changes from a single database, and each publication exists in only one database. This means that you need to create one or more streams per database, with one replication slot per stream. It's also a good practice to keep the publication name aligned with the tables selected in the stream for clarity.

For more information, see Publication and Replication slots in PostgreSQL documentation.

Supported databases

Datastream works with the following types of PostgreSQL databases:

  • AlloyDB for PostgreSQL
  • AlloyDB Omni
  • Amazon Aurora PostgreSQL
  • Amazon RDS for PostgreSQL
  • Cloud SQL for PostgreSQL
  • Self-managed PostgreSQL (on-premise or cloud-hosted)

Datastream also supports replication from PostgreSQL read replica instances for PostgreSQL version 16 and later. For more information, see Replication from read replicas.

What's next