教程:调试将事件路由到 Cloud Run

本教程介绍了如何排查在使用 Eventarc 通过 Cloud Audit Logs 将事件从 Cloud Storage 路由到未经身份验证的 Cloud Run 服务时遇到的运行时错误。

创建 Artifact Registry 标准制品库

创建 Artifact Registry 标准制品库以存储您的容器映像:

gcloud artifacts repositories create REPOSITORY \
    --repository-format=docker \
    --location=$REGION

REPOSITORY 替换为制品库的唯一名称。

创建 Cloud Storage 存储桶

在两个区域中各创建一个 Cloud Storage 存储桶作为 Cloud Run 服务的事件来源:

  1. us-east1 中创建一个存储桶:

    export BUCKET1="troubleshoot-bucket1-PROJECT_ID"
    gcloud storage buckets create gs://${BUCKET1} --location=us-east1
  2. us-west1 中创建一个存储桶:

    export BUCKET2="troubleshoot-bucket2-PROJECT_ID"
    gcloud storage buckets create gs://${BUCKET2} --location=us-west1

创建事件来源后,在 Cloud Run 上部署事件接收器服务。

部署事件接收器

部署接收和记录事件的 Cloud Run 服务。

  1. 通过克隆 GitHub 代码库检索代码示例:

    Go

    git clone https://github.com/GoogleCloudPlatform/golang-samples.git
    cd golang-samples/eventarc/audit_storage
    

    Java

    git clone https://github.com/GoogleCloudPlatform/java-docs-samples.git
    cd java-docs-samples/eventarc/audit-storage
    

    .NET

    git clone https://github.com/GoogleCloudPlatform/dotnet-docs-samples.git
    cd dotnet-docs-samples/eventarc/audit-storage
    

    Node.js

    git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git
    cd nodejs-docs-samples/eventarc/audit-storage
    

    Python

    git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git
    cd python-docs-samples/eventarc/audit-storage
    
  2. 查看本教程的代码,其中包含以下内容:

    • 事件处理脚本,用于接收传入事件作为 HTTP POST 请求中的 CloudEvent:

      Go

      
      // Processes CloudEvents containing Cloud Audit Logs for Cloud Storage
      package main
      
      import (
      	"fmt"
      	"log"
      	"net/http"
      	"os"
      
      	cloudevent "github.com/cloudevents/sdk-go/v2"
      )
      
      // HelloEventsStorage receives and processes a Cloud Audit Log event with Cloud Storage data.
      func HelloEventsStorage(w http.ResponseWriter, r *http.Request) {
      	if r.Method != http.MethodPost {
      		http.Error(w, "Expected HTTP POST request with CloudEvent payload", http.StatusMethodNotAllowed)
      		return
      	}
      
      	event, err := cloudevent.NewEventFromHTTPRequest(r)
      	if err != nil {
      		log.Printf("cloudevent.NewEventFromHTTPRequest: %v", err)
      		http.Error(w, "Failed to create CloudEvent from request.", http.StatusBadRequest)
      		return
      	}
      	s := fmt.Sprintf("Detected change in Cloud Storage bucket: %s", event.Subject())
      	fmt.Fprintln(w, s)
      }
      

      Java

      import io.cloudevents.CloudEvent;
      import io.cloudevents.rw.CloudEventRWException;
      import io.cloudevents.spring.http.CloudEventHttpUtils;
      import org.springframework.http.HttpHeaders;
      import org.springframework.http.HttpStatus;
      import org.springframework.http.ResponseEntity;
      import org.springframework.web.bind.annotation.RequestBody;
      import org.springframework.web.bind.annotation.RequestHeader;
      import org.springframework.web.bind.annotation.RequestMapping;
      import org.springframework.web.bind.annotation.RequestMethod;
      import org.springframework.web.bind.annotation.RestController;
      
      @RestController
      public class EventController {
      
        @RequestMapping(value = "/", method = RequestMethod.POST, consumes = "application/json")
        public ResponseEntity<String> receiveMessage(
            @RequestBody String body, @RequestHeader HttpHeaders headers) {
          CloudEvent event;
          try {
            event =
                CloudEventHttpUtils.fromHttp(headers)
                    .withData(headers.getContentType().toString(), body.getBytes())
                    .build();
          } catch (CloudEventRWException e) {
            return new ResponseEntity<>(e.getMessage(), HttpStatus.BAD_REQUEST);
          }
      
          String ceSubject = event.getSubject();
          String msg = "Detected change in Cloud Storage bucket: " + ceSubject;
          System.out.println(msg);
          return new ResponseEntity<>(msg, HttpStatus.OK);
        }
      }

      .NET

      
      using Microsoft.AspNetCore.Builder;
      using Microsoft.AspNetCore.Hosting;
      using Microsoft.AspNetCore.Http;
      using Microsoft.Extensions.DependencyInjection;
      using Microsoft.Extensions.Hosting;
      using Microsoft.Extensions.Logging;
      
      public class Startup
      {
          public void ConfigureServices(IServiceCollection services)
          {
          }
      
          public void Configure(IApplicationBuilder app, IWebHostEnvironment env, ILogger<Startup> logger)
          {
              if (env.IsDevelopment())
              {
                  app.UseDeveloperExceptionPage();
              }
      
              logger.LogInformation("Service is starting...");
      
              app.UseRouting();
      
              app.UseEndpoints(endpoints =>
              {
                  endpoints.MapPost("/", async context =>
                  {
                      logger.LogInformation("Handling HTTP POST");
      
                      var ceSubject = context.Request.Headers["ce-subject"];
                      logger.LogInformation($"ce-subject: {ceSubject}");
      
                      if (string.IsNullOrEmpty(ceSubject))
                      {
                          context.Response.StatusCode = 400;
                          await context.Response.WriteAsync("Bad Request: expected header Ce-Subject");
                          return;
                      }
      
                      await context.Response.WriteAsync($"GCS CloudEvent type: {ceSubject}");
                  });
              });
          }
      }
      

      Node.js

      const express = require('express');
      const app = express();
      
      app.use(express.json());
      app.post('/', (req, res) => {
        if (!req.header('ce-subject')) {
          return res
            .status(400)
            .send('Bad Request: missing required header: ce-subject');
        }
      
        console.log(
          `Detected change in Cloud Storage bucket: ${req.header('ce-subject')}`
        );
        return res
          .status(200)
          .send(
            `Detected change in Cloud Storage bucket: ${req.header('ce-subject')}`
          );
      });
      
      module.exports = app;

      Python

      @app.route("/", methods=["POST"])
      def index():
          # Create a CloudEvent object from the incoming request
          event = from_http(request.headers, request.data)
          # Gets the GCS bucket name from the CloudEvent
          # Example: "storage.googleapis.com/projects/_/buckets/my-bucket"
          bucket = event.get("subject")
      
          print(f"Detected change in Cloud Storage bucket: {bucket}")
          return (f"Detected change in Cloud Storage bucket: {bucket}", 200)
      
      
    • 使用事件处理脚本的服务器:

      Go

      
      func main() {
      	http.HandleFunc("/", HelloEventsStorage)
      	// Determine port for HTTP service.
      	port := os.Getenv("PORT")
      	if port == "" {
      		port = "8080"
      	}
      	// Start HTTP server.
      	log.Printf("Listening on port %s", port)
      	if err := http.ListenAndServe(":"+port, nil); err != nil {
      		log.Fatal(err)
      	}
      }
      

      Java

      
      import org.springframework.boot.SpringApplication;
      import org.springframework.boot.autoconfigure.SpringBootApplication;
      
      @SpringBootApplication
      public class Application {
        public static void main(String[] args) {
          SpringApplication.run(Application.class, args);
        }
      }

      .NET

          public static void Main(string[] args)
          {
              CreateHostBuilder(args).Build().Run();
          }
          public static IHostBuilder CreateHostBuilder(string[] args)
          {
              var port = Environment.GetEnvironmentVariable("PORT") ?? "8080";
              var url = $"http://0.0.0.0:{port}";
      
              return Host.CreateDefaultBuilder(args)
                  .ConfigureWebHostDefaults(webBuilder =>
                  {
                      webBuilder.UseStartup<Startup>().UseUrls(url);
                  });
          }
      

      Node.js

      const app = require('./app.js');
      const PORT = parseInt(process.env.PORT) || 8080;
      
      app.listen(PORT, () =>
        console.log(`nodejs-events-storage listening on port ${PORT}`)
      );

      Python

      import os
      
      from cloudevents.http import from_http
      
      from flask import Flask, request
      
      app = Flask(__name__)
      if __name__ == "__main__":
          app.run(debug=True, host="0.0.0.0", port=int(os.environ.get("PORT", 8080)))
    • 用于定义服务的运营环境 Dockerfile。Dockerfile 的内容因语言而异。

      Go

      
      # Use the official Go image to create a binary.
      # This is based on Debian and sets the GOPATH to /go.
      # https://hub.docker.com/_/golang
      FROM golang:1.24-bookworm as builder
      
      # Create and change to the app directory.
      WORKDIR /app
      
      # Retrieve application dependencies.
      # This allows the container build to reuse cached dependencies.
      # Expecting to copy go.mod and if present go.sum.
      COPY go.* ./
      RUN go mod download
      
      # Copy local code to the container image.
      COPY . ./
      
      # Build the binary.
      RUN go build -v -o server
      
      # Use the official Debian slim image for a lean production container.
      # https://hub.docker.com/_/debian
      # https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds
      FROM debian:bookworm-slim
      RUN set -x && apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
          ca-certificates && \
          rm -rf /var/lib/apt/lists/*
      
      # Copy the binary to the production image from the builder stage.
      COPY --from=builder /app/server /server
      
      # Run the web service on container startup.
      CMD ["/server"]
      

      Java

      
      # Use the official maven image to create a build artifact.
      # https://hub.docker.com/_/maven
      FROM maven:3-eclipse-temurin-17-alpine as builder
      
      # Copy local code to the container image.
      WORKDIR /app
      COPY pom.xml .
      COPY src ./src
      
      # Build a release artifact.
      RUN mvn package -DskipTests
      
      # Use Eclipse Temurin for base image.
      # https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds
      FROM eclipse-temurin:17.0.16_8-jre-alpine
      
      # Copy the jar to the production image from the builder stage.
      COPY --from=builder /app/target/audit-storage-*.jar /audit-storage.jar
      
      # Run the web service on container startup.
      CMD ["java", "-Djava.security.egd=file:/dev/./urandom", "-jar", "/audit-storage.jar"]
      

      .NET

      
      # Use Microsoft's official build .NET image.
      # https://hub.docker.com/_/microsoft-dotnet-core-sdk/
      FROM mcr.microsoft.com/dotnet/sdk:8.0-alpine AS build
      WORKDIR /app
      
      # Install production dependencies.
      # Copy csproj and restore as distinct layers.
      COPY *.csproj ./
      RUN dotnet restore
      
      # Copy local code to the container image.
      COPY . ./
      WORKDIR /app
      
      # Build a release artifact.
      RUN dotnet publish -c Release -o out
      
      
      # Use Microsoft's official runtime .NET image.
      # https://hub.docker.com/_/microsoft-dotnet-core-aspnet/
      FROM mcr.microsoft.com/dotnet/aspnet:8.0-alpine AS runtime
      WORKDIR /app
      COPY --from=build /app/out ./
      
      # Run the web service on container startup.
      ENTRYPOINT ["dotnet", "AuditStorage.dll"]

      Node.js

      
      # Use the official lightweight Node.js image.
      # https://hub.docker.com/_/node
      FROM node:20-slim
      # Create and change to the app directory.
      WORKDIR /usr/src/app
      
      # Copy application dependency manifests to the container image.
      # A wildcard is used to ensure both package.json AND package-lock.json are copied.
      # Copying this separately prevents re-running npm install on every code change.
      COPY package*.json ./
      
      # Install dependencies.
      # if you need a deterministic and repeatable build create a 
      # package-lock.json file and use npm ci:
      # RUN npm ci --omit=dev
      # if you need to include development dependencies during development
      # of your application, use:
      # RUN npm install --dev
      
      RUN npm install --omit=dev
      
      # Copy local code to the container image.
      COPY . .
      
      # Run the web service on container startup.
      CMD [ "npm", "start" ]
      

      Python

      
      # Use the official Python image.
      # https://hub.docker.com/_/python
      FROM python:3.11-slim
      
      # Allow statements and log messages to immediately appear in the Cloud Run logs
      ENV PYTHONUNBUFFERED True
      
      # Copy application dependency manifests to the container image.
      # Copying this separately prevents re-running pip install on every code change.
      COPY requirements.txt ./
      
      # Install production dependencies.
      RUN pip install -r requirements.txt
      
      # Copy local code to the container image.
      ENV APP_HOME /app
      WORKDIR $APP_HOME
      COPY . ./
      
      # Run the web service on container startup. 
      # Use gunicorn webserver with one worker process and 8 threads.
      # For environments with multiple CPU cores, increase the number of workers
      # to be equal to the cores available.
      CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 main:app

  3. 使用 Cloud Build 构建容器映像,并将映像上传到 Artifact Registry:

    export PROJECT_ID=$(gcloud config get-value project)
    export SERVICE_NAME=troubleshoot-service
    gcloud builds submit --tag $REGION-docker.pkg.dev/${PROJECT_ID}/REPOSITORY/${SERVICE_NAME}:v1
  4. 将容器映像部署到 Cloud Run:

    gcloud run deploy ${SERVICE_NAME} \
        --image $REGION-docker.pkg.dev/${PROJECT_ID}/REPOSITORY/${SERVICE_NAME}:v1 \
        --allow-unauthenticated

    部署成功后,命令行会显示服务网址。

创建触发器

部署 Cloud Run 服务后,设置触发器以通过审核日志监听来自 Cloud Storage 的事件。

  1. 创建 Eventarc 触发器以监听使用 Cloud Audit Logs 路由的 Cloud Storage 事件:

    gcloud eventarc triggers create troubleshoot-trigger \
        --destination-run-service=troubleshoot-service \
        --event-filters="type=google.cloud.audit.log.v1.written" \
        --event-filters="serviceName=storage.googleapis.com" \
        --event-filters="methodName=storage.objects.create" \
        --service-account=${PROJECT_NUMBER}-compute@developer.gserviceaccount.com
    

    这将创建一个名为 troubleshoot-trigger 的触发器。

  2. 要确认 troubleshoot-trigger 已创建,请运行以下命令:

    gcloud eventarc triggers list
    

    输出应类似如下所示:

    NAME: troubleshoot-trigger
    TYPE: google.cloud.audit.log.v1.written
    DESTINATION: Cloud Run service: troubleshoot-service
    ACTIVE: By 20:03:37
    LOCATION: us-central1
    

生成并查看事件

确认您已成功部署服务并可以接收来自 Cloud Storage 的事件。

  1. 创建一个文件并上传到 BUCKET1 存储桶:

     echo "Hello World" > random.txt
     gcloud storage cp random.txt gs://${BUCKET1}/random.txt
    
  2. 监控日志以检查服务是否已收到事件。如需查看日志条目,请完成以下步骤:

    1. 过滤日志条目并以 JSON 格式返回输出:

      gcloud logging read "resource.labels.service_name=troubleshoot-service \
          AND textPayload:random.txt" \
          --format=json
    2. 查找类似如下的日志条目:

      "textPayload": "Detected change in Cloud Storage bucket: ..."
      

请注意,系统最初不会返回任何日志条目。这表示设置中存在问题,您必须进行调查。

调查问题

完成相应过程以调查服务未收到事件的原因。

初始化时间

虽然触发器会立即创建,但触发器最多可能需要两分钟来传播和过滤事件。运行以下命令以确认触发器处于活跃状态:

gcloud eventarc triggers list

输出会指示触发器的状态。在以下示例中,troubleshoot-trigger 将在 14:16:56 之前变为活跃状态:

NAME                  TYPE                               DESTINATION_RUN_SERVICE  ACTIVE
troubleshoot-trigger  google.cloud.audit.log.v1.written  troubleshoot-service     By 14:16:56

触发器处于活跃状态后,再次将文件上传到存储桶。事件会写入 Cloud Run 服务日志。如果服务未收到事件,可能与事件的大小有关。

审核日志

在本教程中,Cloud Storage 事件使用 Cloud Audit Logs 进行路由并发送到 Cloud Run。确认是否为 Cloud Storage 启用了审核日志。

  1. 在 Google Cloud 控制台中,前往审核日志页面。

    转到审核日志

  2. 选中 Google Cloud Storage 复选框。
  3. 确保已选择管理员读取数据读取数据写入日志类型。

启用 Cloud Audit Logs 后,再次将文件上传到存储桶并检查日志。如果服务仍然未收到事件,这可能与触发器位置有关。

触发器位置

不同位置可能有多个资源,您必须过滤来自与 Cloud Run 目标位于同一区域的来源的事件。如需了解详情,请参阅 Eventarc 支持的位置了解 Eventarc 位置

在本教程中,您将 Cloud Run 服务部署到了 us-central1。由于您将 eventarc/location 设置为 us-central1,因此还在同一位置创建了触发器。

但是,您在 us-east1us-west1 位置创建了两个 Cloud Storage 存储桶。如需从这些位置接收事件,您必须在这些位置创建 Eventarc 触发器。

创建位于 us-east1 的 Eventarc 触发器:

  1. 确认现有触发器的位置:

    gcloud eventarc triggers describe troubleshoot-trigger
    
  2. 将位置和区域设置为 us-east1

    gcloud config set eventarc/location us-east1
    gcloud config set run/region us-east1
    
  3. 通过构建容器映像并将其部署到 Cloud Run 来重新部署事件接收器

  4. 创建位于 us-east1 的新触发器:

    gcloud eventarc triggers create troubleshoot-trigger-new \
      --destination-run-service=troubleshoot-service \
      --event-filters="type=google.cloud.audit.log.v1.written" \
      --event-filters="serviceName=storage.googleapis.com" \
      --event-filters="methodName=storage.objects.create" \
      --service-account=${PROJECT_NUMBER}-compute@developer.gserviceaccount.com
    
  5. 确认触发器已创建:

    gcloud eventarc triggers list
    

    触发器最多可能需要两分钟来完成初始化,然后才能开始路由事件。

  6. 如需确认触发器现在已正确部署,请生成并查看事件

您可能遇到的其他问题

使用 Eventarc 时,您可能会遇到其他问题。

事件大小

您发送的事件不得超过事件大小的限制。

之前可传送事件的触发器停止工作

  1. 验证来源正在生成事件。检查 Cloud Audit Logs 并确保受监控服务正在发出日志。如果记录了日志,但事件未被传送,请与支持团队联系

  2. 验证存在与触发器同名的 Pub/Sub 主题。 Eventarc 使用 Pub/Sub 作为其传输层,并且使用现有的 Pub/Sub 主题,或者自动为您创建主题并进行管理。

    1. 如需列出触发器,请参阅 gcloud eventarc triggers list
    2. 如需列出 Pub/Sub 主题,请运行以下命令:

      gcloud pubsub topics list
      
    3. 验证 Pub/Sub 主题名称是否包含已创建的触发器的名称。例如:

      name: projects/PROJECT_ID/topics/eventarc-us-east1-troubleshoot-trigger-new-123

    如果缺少 Pub/Sub 主题,请再次为特定提供商、事件类型和 Cloud Run 目标位置创建触发器

  3. 验证是否已为服务配置触发器。

    1. 在 Google Cloud 控制台中,前往服务页面。

      进入 Service

    2. 点击相应服务的名称,打开其服务详情页面。

    3. 点击触发器标签页。

      系统应该会列出与服务关联的 Eventarc 触发器。

  4. 使用 Pub/Sub 指标类型验证 Pub/Sub 主题和订阅的健康状况。

    • 您可以使用 subscription/dead_letter_message_count 指标监控转发的无法传送的消息。此指标显示 Pub/Sub 从订阅转发的无法传送的消息数量。

      如果消息未发布到主题,请检查 Cloud Audit Logs 并确保受监控服务正在发出日志。如果记录了日志,但事件未被传送,请与支持团队联系

    • 您可以使用 subscription/push_request_count 指标并按 response_codesubcription_id 对指标进行分组,以便监控推送订阅

      如果报告了推送错误,请检查 Cloud Run 服务日志。如果接收端点返回不正常状态代码,则表示 Cloud Run 代码未按预期工作,您必须与支持团队联系

    如需了解详情,请参阅创建指标阈值提醒政策