View flow logs for ULL traffic

This page describes support for ultra-low latency (ULL) unicast and multicast traffic in VPC Flow Logs.

VPC Flow Logs collects packets in ULL Virtual Private Cloud (VPC) networks to generate flow logs. Flow logs are aggregated by IP connection (5-tuple).

These logs can be used for network monitoring, forensics, security analysis, and expense optimization. For example, multicast producers can use log-based metrics to track packet loss for multicast consumers. For more information, see the Log-based metrics overview in the Cloud Logging documentation.

For more information, see the VPC Flow Logs overview.

VPC Flow Logs records for ULL traffic

The following table summarizes the unique information included in VPC Flow Logs records for ULL unicast and multicast traffic, depending on the reporter of the flow and the type of traffic. The information in the table applies to ULL Compute Engine instances.

For a full list of fields in VPC Flow Logs records, see About VPC Flow Logs records.

Flow reporter Description of logs
ULL unicast source or destination instance VPC Flow Logs records include the following additional information:
  • Instance packet loss (for source instances)
  • Network packet loss
  • Network latency
  • Network jitter

For details, see Record format for ULL unicast.
ULL multicast consumer VPC Flow Logs records include the following additional information:
  • Information about the corresponding multicast domain and group range so that you can observe throughput from a multicast group to the multicast consumer.
  • Network packet loss
  • Network latency

For details, see Record format for ULL multicast consumers.

Record format for ULL unicast

The following table describes the unique fields and field format differences in VPC Flow Logs records for ULL unicast traffic.

For a given ULL unicast flow, VPC Flow Logs might generate several different flow log records for the same traffic:

  • One log record for successfully delivered packets. This record is generated by default. The only exceptions are when instance egress packet loss occurs, or in rare cases when 100% network packet loss occurs during the aggregation interval.

  • One or more log records for dropped packets, depending on the drop reason. These records are generated only if packet loss occurs.

For details about the fields in each log record, see the following table.

Field Field format Field type: Base or optional metadata
disposition string
If the log represents packet loss, this field is populated with a value of DROPPED. Otherwise, this field isn't populated. 		Base
drop_reason string
If the value of the disposition field is DROPPED, this field is populated with one of the following values:
  • LOST_IN_TRANSIT: represents network drops
  • SPOOFED_SOURCE: represents instance egress packet loss caused by source IP address spoofing
  • UNREACHABLE_DESTINATION: represents instance egress packet loss caused by an unknown destination
 Base
one_way_network_latencies Latencies
If the disposition isn't populated, meaning there is no packet loss, this field is populated with one-way network latency in milliseconds with nanosecond-level precision as measured during the aggregation interval. The latency measurement excludes time consumed by the application. If not populated, the measurement isn't available. 		Base
one_way_network_latency_jitter LatencyJitter
If the disposition isn't populated, meaning there is no packet loss, this field is populated with one-way network jitter in milliseconds with nanosecond-level precision between the source and destination instance. 		Base
rtt_msec Not populated for ULL unicast traffic. Base
round_trip_time Not populated for ULL unicast traffic. Base
bytes_sent int64
Populated as follows:
  • If the disposition field isn't populated, this field is the number of user payload bytes sent from the source to the destination.
  • If the disposition field is populated with a value of DROPPED and reason LOST_IN_TRANSIT, this field is the number of user payload bytes dropped.
  • If the disposition field is populated with a value of DROPPED and reason SPOOFED_SOURCE or UNREACHABLE_DESTINATION, this field isn't populated.
Consider an example in which a source sends 1000 bytes, but 100 bytes are dropped because of network packet loss. In this case, two separate logs are generated: one with bytes_sent: 900, and one with bytes_sent: 100. 		Base
packets_sent int64
Populated as follows:
  • If the disposition field isn't populated, this field is the number of packets sent from the source to the destination.
  • If the disposition field is populated with a value of DROPPED and reason LOST_IN_TRANSIT, this field is the number of packets dropped.
  • If the disposition field is populated with a value of DROPPED and reason SPOOFED_SOURCE or UNREACHABLE_DESTINATION, this field isn't populated.
 Base

Record format for ULL multicast consumers

The following table describes the unique fields and field format differences in VPC Flow Logs records for multicast traffic when the reporter of the flow is a multicast consumer.

For a given multicast consumer flow, VPC Flow Logs generates up to two separate flow log records for the same traffic:

  • One log record for successfully delivered packets. This record is generated by default, except in rare cases when 100% network packet loss occurs during the aggregation interval.
  • One log record for dropped packets. This record is generated only if packet loss occurs.

For details about the fields in each log record, see the following table.

Field Field format Field type: Base or optional metadata
disposition string
If the log represents packet loss, this field is populated with a value of DROPPED. Otherwise, this field isn't populated. 		Base
drop_reason string
If the value of the disposition field is DROPPED, this field is populated with a value of LOST_IN_TRANSIT. Otherwise, this field isn't populated. 		Base
one_way_network_latencies Latencies
If the disposition isn't populated, meaning there is no packet loss, this field is populated with one-way network latency in milliseconds with nanosecond-level precision as measured during the aggregation interval. The latency measurement excludes time consumed by the application. If not populated, the measurement isn't available. 		Base
rtt_msec Not populated for multicast traffic. Base
round_trip_time Not populated for multicast traffic. Base
bytes_sent int64
Populated as follows:
  • If the disposition field isn't populated, this field is the number of user payload bytes sent from the multicast producer to the consumer.
  • If the disposition field is populated with a value of DROPPED, this field is the number of user payload bytes dropped.
Consider an example in which a multicast producer sends 1000 bytes, but 100 bytes are dropped. In this case, two separate logs are generated: one with bytes_sent: 900, and one with bytes_sent: 100. 		Base
packets_sent int64
Populated as follows:
  • If the disposition field isn't populated, this field is the number of packets sent from the multicast producer to the consumer.
  • If the disposition field is populated with a value of DROPPED, this field is the number of packets dropped.
 Base
Source and destination metadata fields
src_multicast_group_consumer_activation MulticastGroupConsumerActivationDetails
If the destination of the flow is a multicast group IP address and the reporter of the flow is a multicast consumer, then this field is populated with multicast consumer association and group consumer activation details. 		Metadata

Field format reference

This section provides a reference for field formats in flow log records.

For a full list of field formats in VPC Flow Logs records, see About VPC Flow Logs records.

Latencies field format

Field Type Description
median_msec double The median latency as measured during the aggregation interval.

LatencyJitter field format

Field Type Description
median_msec double The median latency jitter.

MulticastGroupConsumerActivationDetails field format

Field Type Description
project_id string The ID of the project that contains the multicast consumer VPC network.
location string The zone of the multicast consumer.
name string The name of the multicast group consumer activation that was created when the multicast consumer VPC network was activated for the group range.
domain_association string The name of the multicast consumer association between the multicast consumer VPC network and the domain.

View flow logs in Cloud Logging

You can view flow logs in Cloud Logging, and you can export logs to any destination that Cloud Logging export supports.

To view flow logs in Cloud Logging, do the following:

  1. If you haven't already, configure VPC Flow Logs. See Configure VPC Flow Logs in the VPC Flow Logs documentation.
  2. Follow the steps to access flow logs.

Analyze traffic flows with Flow Analyzer

Flow Analyzer lets you quickly and efficiently understand your traffic flows without the need to write complex SQL queries. For more information, see the Flow Analyzer overview.

Before you begin, review the following limitations:

  • All of the information in VPC Flow Logs records for ULL traffic is available in Flow Analyzer, except for ULL unicast packet loss.
  • Connectivity Tests doesn't support multicast. In Flow Analyzer, UI options for running Connectivity Tests are disabled where applicable.

To analyze traffic flows, do the following:

  1. If you haven't already, enable Observability Analytics. See Enable Observability Analytics in the Flow Analyzer documentation.

  2. Follow the steps to analyze your traffic flows by building and running queries on your flow logs.

    Consider the following when analyzing ULL unicast and multicast traffic flows:

    • Filters: Basic filters let you define the scope of a query.
    • Display options: After running a query, you can refine your results by using display options.
      • To view throughput data, use the Data volume display mode.
      • To view latency and jitter data, use the Latency display mode.
        • To view latency, ensure that you select One-way (median) for the Metric type. While round-trip time (RTT) is the default metric type, this data isn't available in ULL unicast and multicast flow logs.
        • To view jitter, select One-way jitter for the Metric type.
      • To view packet loss data, use the Packet drops display mode. Packet loss data is available only for multicast traffic. The following options are available for the Packet drops display mode:
        • Metric type: Like the metric types available for data volume, you can choose between Packets sent (default) and Bytes sent.
        • Show timeline: Use this option to determine how the packet loss is calculated and displayed:
          • Drop rate (default): displays the percentage of overall traffic that was dropped.
          • Drop count: displays the number of packets or bytes dropped.
        • Advanced settings, Sampling points: Similar to the sampling point option for data volume, you can choose an approach to determine which flow logs contribute to the display:
          • Source and destination (default): packet loss reported by either endpoint of a flow.
          • Source: packet loss reported by the source endpoint of a flow.
          • Destination: packet loss reported by the destination endpoint of a flow.

        Additionally, when viewing packet loss data, you can click the value in the Drop Reason column of a given flow for more details about the particular occurrence of packet loss and how to mitigate it.

Filter reference for multicast fields

This section provides a reference for the Flow Analyzer filters that correspond to VPC Flow Logs record fields for multicast traffic.

Filters for multicast consumer logs

Field in VPC Flow Logs records Filter in Flow Analyzer Filter type
MulticastGroupConsumerActivationDetails
src_multicast_group_consumer_activation.project_id Multicast Consumer Project ID Flow parameter
src_multicast_group_consumer_activation.location Multicast Consumer Location Flow parameter
src_multicast_group_consumer_activation.name Multicast Consumer Activation Name Flow parameter
src_multicast_group_consumer_activation.domain_association Multicast Consumer Domain Association Flow parameter