Export logs to BigQuery

You can route platform logs generated by Google Cloud Fraud Defense from Cloud Logging to BigQuery by creating a log sink in Log Router.

You can export platform logs to BigQuery to retain assessment data long term, analyze risk scores and reason codes with SQL, and visualize traffic trends in Data Studio. Platform logs in Logging have a default retention period of 30 days, which you can configure to be up to 3,650 days.

Fraud Defense supports platform logging for the following v1 API operations:

Before you begin

To get the permissions that you need to enable platform logs and route them to BigQuery, ask your administrator to grant you the following Identity and Access Management roles on your Google Cloud project:

Enable platform logs

Before routing logs to BigQuery, enable platform logs in Fraud Defense:

  1. In the Google Cloud console, go to the Fraud Defense page.

    Go to Fraud Defense

  2. Verify that your project name appears in the resource selector.

    If you don't see your project name, click the resource selector, and then select your project.

  3. Click Settings.

  4. In the Platform logging card, click Configure.

  5. In the Platform logging pane, click the toggle for Enable assessment logs.

  6. Optional: To export annotation requests, click the toggle next to Enable annotation logs.

  7. Click Save.

For more information about managing platform log settings, see Working with platform logs.

Create a BigQuery sink

After you enable platform logs, create a sink in Log Router to export new reCAPTCHA assessment log entries to BigQuery:

  1. In the Google Cloud console, go to the Log Router page.

    Go to Log Router

  2. Click Create sink.

  3. In the Sink details section, enter a sink name and an optional description, and then click Next.

  4. To configure the destination dataset, complete the following steps in the Sink destination section:

    1. In the Select sink service menu, select BigQuery dataset.
    2. In the Select BigQuery dataset menu, select an existing dataset or select Create new BigQuery dataset.
    3. If you create a new dataset, enter a dataset name, and then click Create dataset.
    4. Select Use partitioned tables, and then click Next.
  5. To include all assessment log entries, in the Choose logs to include in sink section, enter the following inclusion filter in the Build inclusion filter field:

    resource.type="recaptchaenterprise.googleapis.com/Key"
    log_id("recaptchaenterprise.googleapis.com/assessment")
    
  6. Optional: To include both assessment and annotation platform logs in the sink, enter the following filter instead:

    resource.type="recaptchaenterprise.googleapis.com/Key"
    (log_id("recaptchaenterprise.googleapis.com/assessment") OR
      log_id("recaptchaenterprise.googleapis.com/annotation"))
    
  7. Click Next.

  8. Optional: Configure exclusion filters if needed, and then click Create sink.

After the sink is created, Logging displays a confirmation message and automatically provisions a writer service account with permissions to write log entries to your BigQuery dataset.

Pricing

Exporting platform logs to BigQuery incurs standard charges for both Logging and BigQuery:

What's next