Route events across Google Cloud projects

This tutorial shows you how to use Eventarc to read events from a source in one Google Cloud project and route them to a target destination in another Google Cloud project. This is possible by using Pub/Sub as a cross-project transport layer.

Route Pub/Sub events across projects

Because Pub/Sub is a globally distributed service, you can create a topic in one project, publish to that topic from another project, and then trigger Eventarc which routes the message to a Cloud Run service:

1. Set the Google Cloud project ID to your second project:

   gcloud config set project PROJECT_TWO_ID

   Replace PROJECT_TWO_ID with the ID of your second Google Cloud project.

2. In your second project, do the following:

   1. Enable the Cloud Run and Eventarc APIs:

      gcloud services enable run.googleapis.com eventarc.googleapis.com

   2. Set the default location:

      REGION=REGION

      Replace REGION with the supported Eventarc location of your choice. For example, us-central1.

   3. Create a Pub/Sub topic:

      TOPIC=my-topic
      gcloud pubsub topics create $TOPIC

   4. Deploy an unauthenticated Cloud Run service using a prebuilt image, us-docker.pkg.dev/cloudrun/container/hello:

      gcloud run deploy hello \
          --image=us-docker.pkg.dev/cloudrun/container/hello \
          --allow-unauthenticated \
          --region=$REGION

      When you see the service URL, the deployment is complete.

   5. Connect the topic to the service with an Eventarc trigger:

      gcloud eventarc triggers create cross-project-trigger \
          --destination-run-service=hello \
          --destination-run-region=${REGION} \
          --location=${REGION} \
          --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \
          --transport-topic=projects/PROJECT_TWO_ID/topics/$TOPIC

      This creates a trigger called cross-project-trigger.

3. Set the Google Cloud project ID to your first project:

   gcloud config set project PROJECT_ONE_ID

   Replace PROJECT_ONE_ID with the ID of your first Google Cloud project.

4. In your first project, publish a message to the topic in the second project:

   gcloud pubsub topics publish projects/PROJECT_TWO_ID/topics/$TOPIC --message="hello"

5. Set the Google Cloud project ID to your second project:

   gcloud config set project PROJECT_TWO_ID

6. In your second project, confirm that the generated event was logged:

   gcloud logging read "resource.labels.service_name=hello AND jsonPayload.message:hello" --format=json

   A logging entry similar to the following is returned:

   "message": "Received event of type google.cloud.pubsub.topic.v1.messagePublished. Event data: hello"

Route Cloud Storage events across projects

Use Pub/Sub notifications for Cloud Storage to publish events from one project to another, and then route the events to a Cloud Run service through an Eventarc trigger:

1. Set the Google Cloud project ID to your first project:

   gcloud config set project PROJECT_ONE_ID

2. Create a Cloud Storage bucket:

   PROJECT1=$(gcloud config get-value project)
   BUCKET=$PROJECT1-cross-project
   gcloud storage buckets create gs://$BUCKET --location=${REGION}

3. Create a Pub/Sub notification for the bucket to the topic in your second project:

   gcloud storage buckets notifications create gs://$BUCKET --topic=projects/PROJECT_TWO_ID/topics/$TOPIC --payload-format=json

4. Upload a file to the bucket:

   echo "Hello World" > random.txt
   gcloud storage cp random.txt gs://$BUCKET/random.txt

5. Set the Google Cloud project ID to your second project:

   gcloud config set project PROJECT_TWO_ID

6. In your second project, confirm that the generated event was logged:

   gcloud logging read "resource.labels.service_name=hello AND jsonPayload.message:random.txt" --format=json

   A logging entry similar to the following is returned:

   Received event of type google.cloud.pubsub.topic.v1.messagePublished. Event data: {
     "kind": "storage#object",
     "id": "project1-cross-project/random.txt/1635327604259719",
     "selfLink": "https://www.googleapis.com/storage/v1/b/project1-cross-project/o/random.txt",
     "name": "random.txt",
     "bucket": "project1-cross-project",
     "generation": "1635327604259719",
   [...]
   }

Route Cloud Audit Logs events across projects

Requests to your service can be triggered when an audit log entry is created that matches the trigger's filter criteria. (For more information, see Determine event filters for Cloud Audit Logs.) In this case, when a Compute Engine VM instance is created in your first project, an audit log entry that matches the trigger's filter criteria lets you capture and route an event to a Cloud Run service in the second project:

1. Set the Google Cloud project ID to your first project:

   gcloud config set project PROJECT_ONE_ID

2. In your first project, enable the Admin Read, Data Read, and Data Write Log Types for Compute Engine:

   Note that at the project level, you need the roles/owner Identity and Access Management (IAM) role to configure Data Access audit logs for your Google Cloud resources.

   1. Read your project's IAM policy and store it in a file:

      gcloud projects get-iam-policy PROJECT_ONE_ID > /tmp/policy.yaml
      

   2. Edit /tmp/policy.yaml, adding or changing only the Data Access audit logs configuration.

      Caution: You must preserve the bindings: and etag: sections without changes. Failure to do so might cause your project to become unusable.

      auditConfigs:
      - auditLogConfigs:
        - logType: ADMIN_READ
        - logType: DATA_READ
        - logType: DATA_WRITE
        service: compute.googleapis.com
      

   3. Write your new IAM policy:

      gcloud projects set-iam-policy PROJECT_ONE_ID /tmp/policy.yaml
      

      If the preceding command reports a conflict with another change, then repeat these steps, starting with reading the project's IAM policy.

3. In your first project, create a Cloud Logging sink to route Cloud Audit Logs to the topic in your second project:

   gcloud logging sinks create cross-project-sink \
       pubsub.googleapis.com/projects/PROJECT_TWO_ID/topics/my-topic \
       --log-filter='protoPayload.methodName="beta.compute.instances.insert"'

   A reminder similar to the following should be returned:

   Please remember to grant `serviceAccount:[e-mail]` the Pub/Sub Publisher role on the topic.

4. Set the Google Cloud project ID to your second project:

   gcloud config set project PROJECT_TWO_ID

5. In your second project, grant the role to the service account:

   gcloud pubsub topics add-iam-policy-binding my-topic \
       --member=SERVICE_ACCOUNT \
       --role=roles/pubsub.publisher

   Replace SERVICE_ACCOUNT with the service account email address returned in the previous step.

6. Set the Google Cloud project ID to your first project:

   gcloud config set project PROJECT_ONE_ID

7. In your first project, create a Compute Engine VM instance.

   If you are using the Google Cloud console to create the VM instance, you can accept the defaults for the purposes of this tutorial.

8. Set the Google Cloud project ID to your second project:

   gcloud config set project PROJECT_TWO_ID

9. In your second project, confirm that the generated event was logged:

   gcloud logging read "resource.labels.service_name=hello AND jsonPayload.message:beta.compute.instances.insert" --format=json

   A logging entry similar to the following is returned:

   Received event of type google.cloud.pubsub.topic.v1.messagePublished. Eventdata: {
     "logName": "projects/workflows-atamel/logs/cloudaudit.googleapis.com%2Factivity",
     "operation": {
       "id": "operation-1635330842489-5cf5321f4f454-ecc363cd-3883c08d",
       "last": true,
       "producer": "compute.googleapis.com"
     },
     "protoPayload": {
       "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
       "methodName": "beta.compute.instances.insert",
     }
   [...]
   }