Migrate to the OTLP exporter

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

This document describes how to modify a OpenTelemetry Collector logging configuration that uses an existing exporter — in this case, the googlecloud logging exporter — to use the OTLP exporter and the Telemetry (OTLP) API, telemetry.googleapis.com.

Enable the Telemetry API

The OTLP exporter writes to the Telemetry API, so that API must be enabled in your project. Enable the Telemetry API by running the following command:

gcloud services enable telemetry.googleapis.com

Locate the exporter to be replaced

In your configuration file, locate the exporters entry for logging. Your configuration might look like the following:

exporters:
  googlecloud/logging:

Replace googlecloud/logging exporter and add the googleauthclient extension:

exporters:
  otlp_grpc/otlp_logs:
    auth:
      authenticator: googleclientauth
    balancer_name: pick_first
    endpoint: telemetry.googleapis.com:443

extensions:
  googleclientauth: {}

Add processors

In your configuration file, locate the processors entry. Your configuration might look like the following:

processors:
  resourcedetection/_global_0: [...details omitted...]
  transform/mylog__source_0:  [...details omitted...]

Copy and paste the configuration for each of the following processors into your configuration file:

1. A resource processor that captures information about your Google Cloud project.

     resource/gcp_project_id:
       attributes:
         - action: insert
           value: PROJECT_ID
           key: gcp.project_id
   

2. A transform processor that preserves information about the instrumentation source.

     transform/otlp_grpc/preserve_instrumentation_source_version:
       error_mode: ignore
       log_statements:
         - context: log
           statements:
             - set(attributes["instrumentation_source"], instrumentation_scope.name) where instrumentation_scope.name != ""
             - set(attributes["instrumentation_version"], instrumentation_scope.version) where instrumentation_scope.version != ""
   

3. A transform processor that preserves information about the resource attributes.

     transform/otlp_grpc/preserve_service_resource_attributes:
       error_mode: ignore
       log_statements:
         - context: log
           statements:
             - set(attributes["service.name"], resource.attributes["service.name"]) where resource.attributes["service.name"] != nil
             - set(attributes["service.namespace"], resource.attributes["service.namespace"]) where resource.attributes["service.namespace"] != nil
             - set(attributes["service.instance.id"], resource.attributes["service.instance.id"]) where resource.attributes["service.instance.id"] != nil
   

The modified configuration looks like the following:

processors:
  resourcedetection/_global_0: [...details omitted...]
  transform/mylog__source_0:  [...details omitted...]
  resource/gcp_project_id: [...details omitted...]
  transform/otlp_grpc/preserve_instrumentation_source_version: [...details omitted...]
  transform/otlp_grpc/preserve_service_resource_attributes: [...details omitted...]

Modify the logging service and pipeline

In your configuration, locate the service used for logging, which includes a pipeline that uses the googlecloud/logging exporter. Your configuration might look like the following:

service:
  pipelines:
    logs/logs_my__pipeline_mylog__source:
      exporters:
        - googlecloud/logging
      processors:
        - transform/mylog__source_0
        - resourcedetection/_global_0
      receivers:
        - tcplog

Make the following changes to this configuration:

1. Add the googleclientauth extension to the service entry.
2. Change the pipeline's exporter to otlp_grpc/otlp_logs.
3. Add the new processors to the pipeline's processors list:
   • resource/gcp_project_id
   • transform/otlp_grpc/preserve_instrumentation_source_version
   • transform/otlp_grpc/preserve_service_resource_attributes

The modified configuration looks like the following:

service:
  extensions:
    - googleclientauth
  pipelines:
    logs/logs_my__pipeline_mylog__source:
      exporters:
        - otlp_grpc/otlp_logs
      processors:
        - transform/mylog__source_0
        - resourcedetection/_global_0
        - resource/gcp_project_id
        - transform/otlp_grpc/preserve_instrumentation_source_version
        - transform/otlp_grpc/preserve_service_resource_attributes
      receivers:
        - tcplog

Migrate dashboards and alerting policies, if necessary

If you have charts or alerting policies that track usage of the Cloud Logging API, then update them to monitor usage of the Telemetry API instead. The OTLP exporter sends logs to your Google Cloud project by using the Telemetry API instead of the Cloud Logging API.