Configure AlloyDB Omni log rotation

Select a documentation version:

This document describes how to configure rotation of AlloyDB Omni diagnostic logs when you use the AlloyDB Omni Kubernetes operator.

The following log files are located in the /obs/diagnostic/ directory:

  • postgresql.log: This log file collects PostgreSQL server logs, such as client connections, queries, and administrative notices. These logs are always collected and don't need to be enabled.

  • postgresql.audit: This log file collects session and object access audit logs. To collect audit logs, see Manage audit logs with pgAudit.

  • postgresql.internal: AlloyDB Omni provides more messages in addition to regular PostgreSQL logs. These messages provide additional technical information that can assist in the debugging of database issues.

Understand and configure log rotation

Learn how AlloyDB Omni handles log rotation and how to configure rotation thresholds based on file size or duration.

Log rotation behavior

When a log file rotates, the following occurs:

  1. The existing log file is archived into the /obs/diagnostic/archive/ directory.

  2. The active log file in /obs/diagnostic/ is reset so that subsequent log entries begin writing to a clean file.

  3. Log information continues writing to the active log file until the file reaches a size or age threshold, at which point it rotates again. Logs rotate so that active log files don't become too large.

By default, the rotation setting is for each log file to rotate when its size reaches 200 MB. The default rotation doesn't include an age setting.

Archived files are retained for 7 days. Archived files that are older than 7 days are automatically removed, except for the file that was archived during the last rotation. For example, if log_rotation_age is older than 7 days, then the archived file reaches the 7-day threshold before the rotation of the current file. In this case, this archived file isn't removed until the next rotation generates a new archived file.

Each rotated log filename follows the format postgresql-%Y-%m-%d_%H%M%S.EXTENSION.gz, where EXTENSION is log, audit, or internal. The rotation time determines the timestamp in Coordinated Universal Time (UTC). For example, if a log file rotates at 13:01:02 on December 20, 2024 UTC, the archived filename is postgresql-2024-12-20_130102.log.gz.

The system compresses each archived file individually by using the gzip file format.

Configure rotation thresholds

To control when logs rotate, configure a maximum file size, a duration between log rotations, or both. The duration between log rotations is also called the age of the log. If you use both settings, then each log rotates when it reaches one of the thresholds.

To configure log rotation, you set one or both of the following parameters in the parameters section of the DBCluster manifest:

  • log_rotation_size: "SIZE_IN_KB"
  • log_rotation_age: "AGE_IN_MINUTES"

Replace the following:

  • SIZE_IN_KB: the maximum file size, in kilobytes, before logs rotate.
  • AGE_IN_MINUTES: the duration between log rotations, in minutes.

To disable one of the log rotation settings, set it to zero, "0". To retain the default setting that rotates logs when their file size reaches 200 MB, don't set either parameter.

Rotate by log size and duration

The following sample sets logs to rotate when their file size reaches 400 MB or when the time between log rotations reaches one day, whichever happens first:

apiVersion: alloydbomni.dbadmin.goog/v1
kind: DBCluster
metadata:
  name: DB_CLUSTER_NAME
spec:
...
  primarySpec:
  ...
    parameters:
      log_rotation_size: "400000" # 400 MB
      log_rotation_age: "1440" # 24 hours * 60 minutes = 1 day

Replace DB_CLUSTER_NAME with the name of your database cluster.

Rotate by log size only

The following sample sets logs to rotate when their file size reaches 400 MB:

apiVersion: alloydbomni.dbadmin.goog/v1
kind: DBCluster
metadata:
  name: DB_CLUSTER_NAME
spec:
...
  primarySpec:
  ...
    parameters:
      log_rotation_size: "400000" # 400 MB
      log_rotation_age: "0" # Set to 0 to disable

Replace DB_CLUSTER_NAME with the name of your database cluster.

Rotate by duration only

The following sample sets logs to rotate once every 24 hours:

apiVersion: alloydbomni.dbadmin.goog/v1
kind: DBCluster
metadata:
  name: DB_CLUSTER_NAME
spec:
...
  primarySpec:
  ...
    parameters:
      log_rotation_size: "0" # Set to 0 to disable
      log_rotation_age: "1440" # 24 hours * 60 minutes = 1 day

Replace DB_CLUSTER_NAME with the name of your database cluster.

Supported PostgreSQL logging parameters

You can configure what information is emitted to postgresql.log by setting standard PostgreSQL logging Grand Unified Configuration (GUC) parameters in the parameters section of your DBCluster manifest:

apiVersion: alloydbomni.dbadmin.goog/v1
kind: DBCluster
metadata:
  name: DB_CLUSTER_NAME
spec:
...
  primarySpec:
  ...
    parameters:
      log_min_messages: "warning"
      log_min_duration_statement: "250" # Log queries taking >= 250ms
      log_statement: "ddl" # Log all DDL statements
      log_lock_waits: "on"
      log_checkpoints: "on"
      log_temp_files: "0" # Log all temporary files created
      log_autovacuum_min_duration: "1000" # Log autovacuum runs taking >= 1s

Replace DB_CLUSTER_NAME with the name of your database cluster.

The following table describes supported PostgreSQL logging parameters based on Error Reporting and Logging in the PostgreSQL documentation.

Parameter Description
log_min_messages Controls which message levels are written to the server log. Valid values are debug5, debug4, debug3, debug2, debug1, info, notice, warning, error, log, fatal, and panic. Each level includes all following severity levels.
log_min_error_statement Controls which SQL statements that cause an error condition are recorded in the server log. Valid values are debug5 through panic. To disable statement error logging, set to panic.
log_min_duration_statement Logs the duration of each completed statement if the statement ran for at least the specified number of milliseconds. Setting this parameter to 0 logs all statements and durations. Setting it to -1 disables duration logging.
log_min_duration_sample Samples completed statements running for at least the specified milliseconds, at the rate set by log_statement_sample_rate. Setting it to -1 disables sampled duration logging.
log_statement_sample_rate Specifies the fraction of statements exceeding log_min_duration_sample that are logged (0.0 to 1.0).
log_transaction_sample_rate Specifies the fraction of transactions whose statements are all logged (0.0 to 1.0), regardless of statement execution duration.
log_statement Controls which SQL statements are logged: none (off), ddl (all data definition commands), mod (DDL plus data-modifying statements such as INSERT, UPDATE, and DELETE), or all.
log_duration Logs the duration of every completed statement. Query text isn't printed unless the log_min_duration_statement threshold is also exceeded.
log_line_prefix Specifies the printf-style prefix output at the beginning of each log line (such as %m for timestamp, %p for process ID, %u for user, and %d for database).
log_connections Controls whether client connections to the database are logged.
log_disconnections Controls whether session terminations and session durations are logged.
log_hostname Controls whether the connecting client hostname is logged in addition to its IP address.
log_error_verbosity Controls the detail level in server log messages: terse, default, or verbose (includes SQLSTATE code and source lines).
log_lock_waits Controls whether a message is logged when a session waits longer than deadlock_timeout to acquire a lock.
log_checkpoints Records checkpoints and restartpoints in the log, including buffer write statistics and timing.
log_autovacuum_min_duration Logs each action executed by autovacuum if it ran for at least the specified number of milliseconds. Setting this parameter to 0 logs all actions. Setting it to -1 disables logging.
log_temp_files Controls logging of temporary file names and sizes in kilobytes when deleted (for sorts, hashes, or query results). Setting this parameter to 0 logs all temporary files. Setting it to -1 disables logging.
log_parameter_max_length Trims bind parameter values logged in non-error statement logs to this number of bytes. Setting this parameter to -1 logs values in full. Setting it to 0 disables parameter logging.
log_parameter_max_length_on_error Trims bind parameter values reported in error messages to this number of bytes. Setting this parameter to 0 disables parameters in error messages. Setting it to -1 logs values in full.
log_replication_commands Controls whether replication commands and replication slot actions are logged.
log_recovery_conflict_waits Controls whether a message is logged when the startup process waits longer than deadlock_timeout for recovery conflicts.
log_startup_progress_interval Sets the time in milliseconds between progress messages for long-running startup operations. Setting this parameter to 0 disables it.
log_timezone Sets the time zone used for timestamps in the server log.

Manage audit logs with pgAudit

To enable session and object access logging, you must configure pgAudit parameters in your database cluster and install the extension in your databases.

Step 1: Configure pgAudit parameters in the DBCluster manifest

To collect session and object access logs in the postgresql.audit file, enable pgAudit and configure which statements to log by using the pgaudit.log parameter.

Add the following lines to the parameters section of the v1_dbcluster_parameters.yaml file:

alloydb.enable_pgaudit: "on"
pgaudit.log: "all"

The following example shows this configuration in the DBCluster manifest:

apiVersion: alloydbomni.dbadmin.goog/v1
kind: DBCluster
metadata:
  name: DB_CLUSTER_NAME
spec:
  databaseVersion: "15.7.1"
  primarySpec:
    ...
    parameters:
      ...
      alloydb.enable_pgaudit: "on"
      pgaudit.log: "all"

Replace DB_CLUSTER_NAME with the name of your database cluster.

For more information, see pgaudit in Supported database extensions.

Step 2: Install the pgAudit extension in each database

After you configure the parameters and the database restarts, connect to your database and install the pgAudit extension:

CREATE EXTENSION IF NOT EXISTS pgaudit;

Run this command in every database where you want to enable audit logging. This command installs the event triggers required for DDL auditing.

Supported pgAudit parameters

The following table describes key pgAudit parameters that you can configure in the parameters section of your DBCluster spec. For more information, see the pgAudit documentation.

Parameter Description Default value
pgaudit.log Specifies which classes of statements are logged by session audit logging. Valid values are none, all, read, write, function, role, ddl, misc, and misc_set. none
pgaudit.log_catalog Specifies that session logging is enabled if all relations in a statement are in pg_catalog. Disabling this reduces log noise from database clients. on
pgaudit.log_parameter Specifies that audit logging includes the parameters passed with the statement. off
pgaudit.log_relation Specifies whether session audit creates a separate log entry for each relation (such as tables and views) referenced in a SELECT or DML statement. off
pgaudit.log_rows Specifies that audit logging includes the number of rows retrieved or affected by a statement. off
pgaudit.log_statement Specifies whether logging includes the statement text and parameters. on
pgaudit.role Specifies the primary role to use for object audit logging. None

PostgreSQL server logs are always collected in the postgresql.log file and don't require enabling pgAudit.

Retrieve the active audit log path

Use the SQL function alloydb_audit_current_logfile to view the path of the audit log file. If auditing is disabled, then the result is NULL.

SELECT alloydb_audit_current_logfile();

 alloydb_audit_current_logfile
----------------------------------
 /obs/diagnostic/postgresql.audit

Forward database logs to a centralized sink

AlloyDB Omni writes database server logs (postgresql.log), audit logs (postgresql.audit), and internal logs (postgresql.internal) to the /obs/diagnostic/ directory on the obsdisk volume inside the database container.

To forward these logs to a centralized logging sink (such as Cloud Logging, Datadog, Splunk, or Elasticsearch), you can deploy a sidecar container by using the Sidecar Custom Resource (CR).

The sidecar container runs in the same Pod as the database, mounts the log volume (obsdisk), and tails the log files to forward them to your logging destination.

Example: Forward logs by using a Fluent Bit sidecar

The following example demonstrates how to use a Fluent Bit sidecar to tail all AlloyDB Omni log files and write them to standard output. Standard Kubernetes cluster log collectors, such as the Logging agent on Google Kubernetes Engine (GKE), can then collect these logs.

  1. Create a ConfigMap containing the Fluent Bit configuration to tail server, audit, and internal log files:

    apiVersion: v1
kind: ConfigMap
metadata:
  name: fluentbit-config
  namespace: NAMESPACE
data:
  fluent-bit.conf: |
    [SERVICE]
        Flush        1
        Daemon       Off
        Log_Level    info

    [INPUT]
        Name         tail
        Path         /obs/diagnostic/postgresql.log,/obs/diagnostic/postgresql.audit,/obs/diagnostic/postgresql.internal
        Tag          alloydb.*
        Path_Key     filename
        DB           /tmp/fluent-bit.db

    [OUTPUT]
        Name         stdout
        Match        *

    Replace NAMESPACE with the namespace of your database cluster.

  2. Create a Sidecar manifest that defines the Fluent Bit container and mounts both the obsdisk volume (containing AlloyDB Omni logs) and the ConfigMap volume:

    apiVersion: alloydbomni.dbadmin.goog/v1
kind: Sidecar
metadata:
  name: logging-forwarder
  namespace: NAMESPACE
spec:
  additionalVolumes:
  - name: fluentbit-config-volume
    configMap:
      name: fluentbit-config
  sidecars:
  - name: fluent-bit
    image: fluent/fluent-bit:3.0
    command: ["/fluent-bit/bin/fluent-bit"]
    args: ["-c", "/fluent-bit/etc/fluent-bit.conf"]
    volumeMounts:
    - name: obsdisk
      mountPath: /obs
    - name: fluentbit-config-volume
      mountPath: /fluent-bit/etc/

    Replace NAMESPACE with the namespace of your database cluster.

  3. Update your DBCluster manifest to reference the sidecar, or patch the cluster by using the following command:

    kubectl patch dbclusters.alloydbomni.dbadmin.goog DB_CLUSTER_NAME -n NAMESPACE -p '{"spec":{"primarySpec":{"sidecarRef":{"name":"logging-forwarder"}}}}' --type=merge

    Replace the following:

    • DB_CLUSTER_NAME: the name of your database cluster.
    • NAMESPACE: the namespace of your database cluster.

    For more information on managing sidecars, see Configure a sidecar container.

Query log rotation history with SQL

When log files rotate, the operator stores the archived files in the /obs/diagnostic/archive/ directory with their rotation timestamps formatted as postgresql-%Y-%m-%d_%H%M%S.EXTENSION.gz.

You can query the latest log rotation time directly from the database by using standard PostgreSQL file system functions.

Inspect rotation timestamps from archived log filenames

To view the most recent log rotations and their parsed UTC rotation timestamps, run the following query:

SELECT
  file,
  to_timestamp(
    substring(file from 'postgresql-([0-9]{4}-[0-9]{2}-[0-9]{2}_[0-9]{6})') || '+00',
    'YYYY-MM-DD_HH24MISS+TZH'
  ) AS rotated_at
FROM pg_ls_dir('/obs/diagnostic/archive') AS file
WHERE file ~ '^postgresql-[0-9]{4}-[0-9]{2}-[0-9]{2}_[0-9]{6}'
ORDER BY rotated_at DESC
LIMIT 5;

The output is similar to the following:

                   file                   |       rotated_at
------------------------------------------+------------------------
 postgresql-2026-09-29_171900.internal.gz | 2026-09-29 17:19:00+00
 postgresql-2026-09-29_072800.log.gz      | 2026-09-29 07:28:00+00
 postgresql-2026-09-28_175300.log.gz      | 2026-09-28 17:53:00+00
 postgresql-2026-09-28_093400.log.gz      | 2026-09-28 09:34:00+00
 postgresql-2026-09-28_092000.audit.gz    | 2026-09-28 09:20:00+00
(5 rows)

What's next