为 Compute Engine 虚拟机配置准确的时间

许多依赖于事件的精密排序的软件系统都依靠稳定、一致的系统时钟,并使用带有时间戳的系统日志来确保时间同步和调试问题。为帮助确保系统时钟同步,Compute Engine 虚拟机 (VM) 实例已预先配置为使用网络时间协议 (NTP),这是时间同步硬件和软件的捆绑式解决方案。如果您非常重视确保时间同步的准确性并监控时间同步的准确性,则可以配置精确时间,而不是 NTP,以使用 chronyptp_kvm 将虚拟机的时钟与宿主时钟同步。此配置旨在使支持的设置达到 1 毫秒内的准确度。

支持的机器类型

以下机器类型支持精确时间:

支持的操作系统

以下操作系统支持精确时间:

操作系统 版本 图片
CentOS Stream 9 centos-stream-9
Container-Optimized OS COS 105 LTS、COS 109 LTS、COS 113 LTS、COS 117 LTS cos-105-lts、cos-109-lts、cos-113-lts、cos-117-lts
Debian 11 (Bullseye)、12 (Bookworm) debian-11, debian-12
Fedora Cloud 39 fedora-cloud-39
RHEL 8, 9 rhel-8-4-sap-ha、rhel-8-6-sap-ha、rhel-8-8-sap-ha、rhel-8-10-sap-ha、rhel-9、rhel-9-0-sap-ha、rhel-9-2-sap-ha、rhel-9-4-sap-ha
Rocky Linux 8, 9 rocky-linux-8、rocky-linux-8-optimized-gcp、rocky-linux-9-optimized-gcp、rocky-linux-9-optimized-gcp
SLES 15 sles-15、sles-15-sp2-byos、sles-15-sp2-sap、sles-15-sp3-byos、sles-15-sp3-sap、sles-15-sp4-byos、sles-15-sp4-sap、sles-15-sp5-byos、sles-15-sp5-sap
Ubuntu 22.04 LTS (Jammy Jellyfish)、24.04 LTS (Noble Numbat) ubuntu-2204-lts、ubuntu-2404-lts-amd64
Ubuntu Pro 2004 ubuntu-pro-2004-lts、ubuntu-pro-2004-lts-amd64

支持的可用区

以下时区支持精确时间:

可用区 位置
europe-west1-b 欧洲比利时圣吉斯兰
europe-west1-c 欧洲比利时圣吉斯兰
europe-west2-b 欧洲英国伦敦
europe-west3-a 欧洲德国法兰克福
us-central1-a 北美洲爱荷华州康瑟布拉夫斯
us-central1-b 北美洲爱荷华州康瑟布拉夫斯
us-central1-c 北美洲爱荷华州康瑟布拉夫斯
us-central1-f 北美洲爱荷华州康瑟布拉夫斯
us-east1-b 北美洲南卡罗来纳州蒙克斯科纳
us-east1-c 北美洲南卡罗来纳州蒙克斯科纳
us-east4-c 北美洲弗吉尼亚阿什本
us-east5-a 北美洲俄亥俄州哥伦布
us-south1-a 北美洲德克萨斯州达拉斯
us-west1-b 北美洲俄勒冈州达尔斯
us-west2-a 北美洲加利福尼亚州洛杉矶
us-west3-a 北美洲犹他州盐湖城

配置精确的时间同步

如需为项目的虚拟机配置精确的时间同步,请针对每个虚拟机完成以下任务:

  • 配置 chrony 以将 ptp-kvm 用作其时间源。
  • 配置 Google Cloud Ops Agent 以进行数据收集和分析。

完成这两项任务后,您项目中的虚拟机便会设置精确的时间同步。

如需查看创建虚拟机并完成这两项任务以配置精确时间同步的示例脚本,请参阅 GitHub 中的虚拟机创建脚本

配置 chrony 以使用 ptp-kvm

如需将 chrony 配置为使用 ptp-kvm 作为其时间源,请在每个 Google Cloud 项目的虚拟机中运行以下脚本:

#!/bin/bash

# Install chrony as needed
if ! command -v chronyc &>/dev/null; then
    # Detect the package manager and install chrony
    if command -v apt &>/dev/null; then
        # Debian, Ubuntu, and derivatives
        echo "Detected apt. Installing chrony..."
        apt-get update
        apt-get install -y chrony
    elif command -v dnf &>/dev/null; then
        # Fedora, RHEL 8+, CentOS 8+
        echo "Detected dnf. Installing chrony..."
        dnf install -y chrony
    elif command -v yum &>/dev/null; then
        # RHEL 7, CentOS 7
        echo "Detected yum. Installing chrony..."
        yum install -y chrony
    elif command -v zypper &>/dev/null; then
        # openSUSE, SLES
        echo "Detected zypper. Installing chrony..."
        zypper install -y chrony
    else
        echo "Please install chrony manually."
        exit 1
    fi
fi

# Different distros place chrony config in
# different locations, detect this.
if [ -f "/etc/chrony/chrony.conf" ]; then
    CHRONY_CONF="/etc/chrony/chrony.conf"
else
    CHRONY_CONF="/etc/chrony.conf"
fi

# Load PTP-KVM clock for high-accuracy clock synchronization
# PTP-KVM allows the VM to read a cross time-stamp of a platform
# provided clock and the VM CPU Clock (CycleCounter), providing
# resiliency from network and virtualization variability when
# synchronizing the realtime/wall clock
/sbin/modprobe ptp_kvm
echo "ptp_kvm" >/etc/modules-load.d/ptp_kvm.conf

# NTP servers might indicate the wrong time due to chrony
# greatly reducing the polling frequency, resulting in
# overall negative impact to the clock synchronization
# achieved, especially after live migration events.
#
# We disable NTP servers to prevent these issues.
#
# Customers interested in monitoring the clock accuracy compared to NTP sources
# should run a second instance of chrony in a no-change mode to do so.

# Disable NTP servers:
sed -i '/^server /d' $CHRONY_CONF
sed -i '/^pool /d' $CHRONY_CONF
sed -i '/^include /d' $CHRONY_CONF

#Disable DHCP based NTP config:
sed -i 's/^NETCONFIG_NTP_POLICY="auto"/NETCONFIG_NTP_POLICY=""/' /etc/sysconfig/network/config
truncate -s 0 /var/run/netconfig/chrony.servers
echo PEERNTP=no | sudo tee -a /etc/sysconfig/network
service NetworkManager restart
systemctl disable systemd-timesyncd.service
timedatectl set-ntp false

# Configure PTP-KVM based HW refclock, with leap second smearing
# Google's clocks are doing leap second smearing, and therefor chrony shouldn't attempt to adjust
# the time received from PTP-KVM to adjust for leap seconds.
sed "s/^leapsectz/#leapsectz/" -i $CHRONY_CONF
echo "refclock PHC /dev/ptp_kvm poll -1" >>$CHRONY_CONF
# For extra debugging logging, uncomment the following line
#echo "log measurements statistics tracking" >> $CHRONY_CONF
# Enable chrony's clock accuracy tracking log,
# for monitoring and auditing.
echo "log tracking" >>$CHRONY_CONF

# Restart chrony (Ubuntu named it differently)
systemctl restart chronyd
systemctl restart chrony

在虚拟机上配置 Google Cloud Ops Agent

如需配置 Google Cloud Ops Agent 以进行数据收集和分析,请在每个 Google Cloud 项目的虚拟机中运行以下脚本:

#!/bin/bash

# From https://cloud.google.com/stackdriver/docs/solutions/agents/ops-agent/installation#install-latest-version
curl -sSO https://dl.google.com/cloudagents/add-google-cloud-ops-agent-repo.sh
bash add-google-cloud-ops-agent-repo.sh --also-install

OPS_AGENT_CONF="
logging:
    receivers:
        chrony_tracking_receiver:
            type: files
            include_paths:
                - /var/log/chrony/tracking.log
    processors:
        chrony_tracking_processor:
            type: parse_regex
            regex: \"^.*PHC0.*  (?<max_error>[-\d\.eE]+)$\"
    service:
        pipelines:
            chrony_tracking_pipeline:
                receivers: [chrony_tracking_receiver]
                processors: [chrony_tracking_processor]
"

OPS_AGENT_CONF_PATH="/etc/google-cloud-ops-agent/config.yaml"
echo "$OPS_AGENT_CONF" >"$OPS_AGENT_CONF_PATH"
systemctl restart google-cloud-ops-agent

配置时间同步监控

如需为 Google Cloud项目的虚拟机配置时间同步监控,请为Google Cloud 项目运行日志记录和信息中心设置脚本。此脚本可帮助您完成 Google Cloud 项目的以下任务:

  • 它会为与虚拟机的 Google Cloud 项目关联的服务账号设置适当的权限。
  • 它会创建一个基于日志的指标,chrony 用于确保虚拟机及其宿主服务器上的时钟保持一致。
  • 它会创建一个信息中心,通过组合以下指标来衡量虚拟机时钟对 UTC 的可追溯性:
    • 虚拟机主机时钟相对于世界协调时间 (UTC) 的准确度,以Google Cloud 指标(即 instance/clock_accuracy/ptp_kvm/nanosecond_accuracy)的形式提供。
    • 用于衡量虚拟机时钟相对于宿主机时钟的准确性的 chrony 指标。

如需完成上述任务,请运行以下脚本:

#!/bin/bash

if [ -z "$1" ]; then
    echo "Usage: time-sync-logging-dashboard.sh <project_id>" >&2
    exit 1
fi

PROJECT_ID="$1"
PROJECT_NUMBER=$(gcloud projects describe "$PROJECT_ID" --format="value(projectNumber)")
SERVICE_ACCOUNT_EMAIL=${PROJECT_NUMBER}-compute@developer.gserviceaccount.com

gcloud projects add-iam-policy-binding "${PROJECT_ID}" \
    --member="serviceAccount:${SERVICE_ACCOUNT_EMAIL}" \
    --role="roles/compute.instanceAdmin"

gcloud projects add-iam-policy-binding "${PROJECT_ID}" \
    --member="serviceAccount:${SERVICE_ACCOUNT_EMAIL}" \
    --role="roles/monitoring.metricWriter"

gcloud projects add-iam-policy-binding "${PROJECT_ID}" \
    --member="serviceAccount:${SERVICE_ACCOUNT_EMAIL}" \
    --role="roles/logging.logWriter"

cp clock-error-metric.json /tmp/clock-error-metric.json
sed -i "s/PROJECT_ID/${PROJECT_ID}/" /tmp/clock-error-metric.json

gcloud logging metrics create --project "${PROJECT_ID}" phc-clock-max-error-gce --config-from-file=/tmp/clock-error-metric.json
gcloud monitoring dashboards create --project "${PROJECT_ID}" --config-from-file=metric-dashboard.json

脚本运行完毕后,使用其创建的信息中心查看项目虚拟机的时钟准确度数据。

后续步骤