Configure a self-managed MongoDB database for CDC

You can stream change data capture (CDC) updates from self-managed MongoDB replica sets or sharded clusters to a supported destination, such as BigQuery or Cloud Storage, by using Datastream. To prepare your source, you create a Datastream user with read privileges in the admin database for your deployment.

Replica set

For information about deploying a MongoDB replica set, see Deploy a self-managed replica set in the MongoDB documentation.

To configure Datastream for use with a self-managed MongoDB replica set, follow these steps:

1. Install the MongoDB Shell. For information about installing the MongoDB Shell, see Install mongosh.
2. Open a terminal window, and run the mongosh command to connect to your replica set or primary node.

3. Create a user for Datastream in the admin database:

   use admin
   db.createUser({
     user: "USERNAME",
     pwd: "PASSWORD",
     roles: [ "readAnyDatabase", {role: "read", db: "admin"} ]
   })
   

   Replace USERNAME and PASSWORD with your preferred username and password.

Sharded cluster

For information about deploying a sharded cluster, see Deploy a sharded cluster in the MongoDB documentation.

To configure Datastream for use with a self-managed MongoDB sharded cluster, follow these steps:

1. Install the MongoDB Shell. For information about installing the MongoDB Shell, see Install mongosh.
2. Open a terminal window, and run the mongosh command to connect to your MongoDB sharded cluster.

3. Create a Datastream user in every primary shard node and the mongos query router:

   use admin
   db.createUser({
      user: "USERNAME",
      pwd: "PASSWORD",
      roles: [ "readAnyDatabase" ]
   })
   

   Replace USERNAME and PASSWORD with your username and password. You can choose any username and password, but they must be identical across all primary shard nodes and the mongos query router.

4. To find out which shards are connected to the mongos query router, run the following command:

   db.adminCommand({ listShards: 1 })