Install the Ops Agent by using VM Extension Manager policies

VM Extension Manager policies let you install the Ops Agent on a fleet of Compute Engine VMs. An extension policy can install the Ops Agent globally across your Google Cloud project or within a specific zone, keep the agent version updated, and apply custom configuration to the agent.

Before you begin

Before creating VM Extension Manager extension policies to manage the Ops Agent, do the following:

Remove existing observability agents

Before creating a VM Extension Manager policy for the Ops Agent, ensure that there are no instances of the Ops Agent or the legacy Monitoring agent and Logging agent already on the VM. Running the Ops Agent and the legacy agents on the same VM can cause ingestion of duplicate logs or conflicts in metrics ingestion.

If you use an extension policy to install the Ops Agent on a VM where another observability agent is already installed, then the policy installs the agent, but the agent fails to run because a conflict is detected.

How you remove existing agents depends on how those agents were installed.

Enable the APIs required to use the Ops Agent

To use the Ops Agent to write logs and metrics, you must enable the Cloud Logging API and Cloud Monitoring API on your Google Cloud project.

Roles required to enable APIs

To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

Enable the APIs

Grant roles required to use the Ops Agent

To get the permissions that you need to use the Ops Agent to write logs and metrics, ask your administrator to grant you the following IAM roles on your service account:

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

For more information about roles and the Ops Agent, see Authorize the Ops Agent.

Enable the API used by VM Extension Manager

VM Extension Manager uses the Compute Engine API. Enable this API in your Google Cloud project if it's not already enabled.

Enable Compute Engine API

Grant roles required to use VM Extension Manager

To get the permissions that you need to create VM extension policies, ask your administrator to grant you the following IAM roles on the project:

For more information about granting roles, see Manage access to projects, folders, and organizations.

These predefined roles contain the permissions required to create VM extension policies. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to create VM extension policies:

  • To create extension policies: compute.vmExtensionPolicies.create

You might also be able to get these permissions with custom roles or other predefined roles.

For more information about IAM roles and permissions in Compute Engine, see Compute Engine roles and permissions.

Install the Google Cloud CLI

You can create and manage VM extension policies by using the Google Cloud console or the Google Cloud CLI, gcloud. To use gcloud, you must first install the Google Cloud CLI.

To see the components you have installed, run the following command:

gcloud components list

To verify that you have the latest version, run the following command:

gcloud components update

Policy conflicts and priorities

VM Extension Manager associates a priorty with each new policy, with lower numbers indicating higher priorities; the default priority is 1000. A VM can be covered by multiple policies for an extension like the Ops Agent, but only one policy for the Ops Agent extension is effective on the VM.

VM Extension Manager uses priorities to resolve conflicts among policies applied to a VM. For more information about the resolution process, see Policy priority and conflict resolution.

Policy rollout plans for global policies

VM Extension Manager global policies use a rollout plan to determine how a policy is distributed to target VMs. There are two types of rollout plans:

  • Predefined rollout plans. There are two predefined rollout plans, chosen by the value used with the --rollout-predefined-plan option for gcloud CLI commands.

    • Slow rollout, which deploys the Ops Agent gradually across different zones over a period of time. This rollout plan is recommended.

      Specify this rollout plan by using the --rollout-predefined-plan=slow_rollout option.

    • Fast rollout, which deploys the Ops Agent to all targeted VMs across all zones and regions immediately.

      Specify this rollout plan by using the --rollout-predefined-plan=fast_rollout option.

  • Custom rollout plans, specified by using the --rollout-custom-plan option for gcloud CLI commands.

    Create custom rollout plans by adding them to the set of rollout plans in your Google Cloud project; for more information, see the Compute Engine API method rolloutPlans.insert.

For more information about rollout plans, see Rollout plans for global policies.

Create VM extension policies to install the Ops Agent

With VM Extension Manager, you create policies that declare which extensions you want to install on VMs. The Ops Agent is one of the available extensions. By defining a policy, you can ensure that supported extensions, including the Ops Agent, are installed and maintained on any VMs that match criteria you specify, such as VM labels.

You can use extension policies for the Ops Agent to perform tasks like the following:

  • Fleetwide installation: Install the Ops Agent on the following:

    • All VMs in one or more zones in your Google Cloud project.
    • All VMs globally (in your Google Cloud project).
    • A subset of VMs in a zone, identified by labels.
    • A subset of VMs across zones in your Google Cloud project, identified by labels.

  • Version control:

    • Keep the Ops Agent updated to the latest version.
    • Pin the Ops Agent version to a specific release.

  • Configuration management: Apply a custom configuration to the Ops Agent on all VMs managed by a policy.

The following sections provide examples of policies that install the Ops Agent according to specific criteria.

VM Extension Manager supports Ops Agent version 2.58.0 or newer, when the Ops Agent has been installed by an extension policy. VM Extension Manager can't manage earlier versions of the Ops Agent, Ops Agent instances installed by using other means, or the legacy Monitoring agent or Logging agent.

Install the latest Ops Agent on all VMs

To create a policy that installs the latest version of the Ops Agent on all VMs in a zone or globally, and updates the Ops Agent when a new version is released, do the following:

Console

To create a zonal policy, you can use the Google Cloud console.

  1. In the Google Cloud console, go to the Extension policies page:

    Go to Extension policies

    If you use the search bar to find this page, then select the result whose subheading is Compute Engine.

  2. Click + Create.
  3. In the Name field, enter a name for the policy.
  4. Optional: In the Description field, enter a description for the policy.
  5. In the Priority field, specify a priority number to resolve conflicts between policies. Lower numbers indicate higher priority. The default value is 1000.
  6. In the Zone list, select the zone where you want to apply this policy.
  7. In the Manage extensions section, click Add extension and do the following:
    1. From the Extension list, select Google Cloud's Extension for Ops Agent.
    2. Leave the Version field blank.
  8. Click Create.

gcloud

To create a zonal or global policy, you can use the gcloud CLI.

Zonal

Use the following gcloud compute zone-vm-extension-policies create command to create the policy:

gcloud compute zone-vm-extension-policies create POLICY_NAME \
  --project=PROJECT_ID \
  --zone=ZONE \
  --extensions=ops-agent

Replace the following variables in the command:

  • POLICY_NAME: A name for the VM extension policy.
  • PROJECT_ID: The identifier of the project.
  • ZONE: The target zone; for example, us-central1-f.

The gcloud compute zone-vm-extension-policies create command takes an option --version option. If you omit the --version option, then the policy installs the latest version and updates the Ops Agent when a new version is released.

Global

Use the following gcloud compute global-vm-extension-policies create command to create the policy:

gcloud compute global-vm-extension-policies create POLICY_NAME \
  --project=PROJECT_ID \
  --extensions=ops-agent  \
  --rollout-predefined-plan=slow_rollout

Replace the following variables in the command:

  • POLICY_NAME: A name for the VM extension policy.
  • PROJECT_ID: The identifier of the project.

The gcloud compute global-vm-extension-policies create command takes a --version option. If you omit the --version option, then the policy installs the latest version and updates the Ops Agent when a new version is released.

Pin the Ops Agent to a specific version on all VMs

To create a policy that installs a specific version of the Ops Agent on all VMs in a zone or globally, do the following:

Console

To create a zonal policy, you can use the Google Cloud console.

  1. In the Google Cloud console, go to the Extension policies page:

    Go to Extension policies

    If you use the search bar to find this page, then select the result whose subheading is Compute Engine.

  2. Click + Create.
  3. In the Name field, enter a name for the policy.
  4. Optional: In the Description field, enter a description for the policy.
  5. In the Priority field, specify a priority number to resolve conflicts between policies. Lower numbers indicate higher priority. The default value is 1000.
  6. In the Zone list, select the zone where you want to apply this policy.
  7. In the Manage extensions section, click Add extension and do the following:
    1. From the Extension list, select Google Cloud's Extension for Ops Agent.
    2. From the Version list, select the version to install. Use version 2.58.0 or newer.
  8. Click Create.

gcloud

To create a zonal or global policy, you can use the gcloud CLI.

Zonal

Use the following gcloud compute zone-vm-extension-policies create command to create the policy:

gcloud compute zone-vm-extension-policies create POLICY_NAME \
  --project=PROJECT_ID \
  --zone=ZONE \
  --extensions=ops-agent  \
  --version=ops-agent=VERSION

Replace the following variables in the command:

  • POLICY_NAME: A name for the VM extension policy.
  • PROJECT_ID: The identifier of the project.
  • ZONE: The target zone; for example, us-central1-f.
  • VERSION: The version of the Ops Agent to install. Use version 2.58.0 or newer.

    If you omit the --version option, then the policy installs the latest version and updates the Ops Agent when a new version is released, as shown in the example that installs the latest version.

Global

Use the following gcloud compute global-vm-extension-policies create command to create the policy:

gcloud compute global-vm-extension-policies create POLICY_NAME \
  --project=PROJECT_ID \
  --extensions=ops-agent  \
  --rollout-predefined-plan=slow_rollout \
  --version=ops-agent=VERSION

Replace the following variables in the command:

  • POLICY_NAME: A name for the VM extension policy.
  • PROJECT_ID: The identifier of the project.
  • VERSION: The version of the Ops Agent to install. Use version 2.58.0 or newer.

    If you omit the --version option, then the policy installs the latest version and updates the Ops Agent when a new version is released, as shown in the example that installs the latest version.

For a list of Ops Agent versions, see the Ops Agent releases page on GitHub.

Install the Ops Agent on VMs that have a specific label

To create a policy that installs the latest version of Ops Agent on the VMs in a zone or globally that have a specific label, do the following:

Console

To create a zonal policy, you can use the Google Cloud console.

  1. In the Google Cloud console, go to the Extension policies page:

    Go to Extension policies

    If you use the search bar to find this page, then select the result whose subheading is Compute Engine.

  2. Click + Create.
  3. In the Name field, enter a name for the policy.
  4. Optional: In the Description field, enter a description for the policy.
  5. In the Priority field, specify a priority number to resolve conflicts between policies. Lower numbers indicate higher priority. The default value is 1000.
  6. In the Zone list, select the zone where you want to apply this policy.
  7. In the Manage extensions section, click Add extension and do the following:
    1. From the Extension list, select Google Cloud's Extension for Ops Agent.
    2. Leave the Version field blank.
  8. In the Target VM instances section, select the VMs for the policy. To select VMs with specific labels, click Add labels and add the key-value pair.
  9. Click Create.

gcloud

To create a zonal or global policy, you can use the gcloud CLI.

Zonal

Use the --inclusion-labels option to the gcloud compute zone-vm-extension-policies create command to specify a comma-separated list of key-value pairs:

gcloud compute zone-vm-extension-policies create POLICY_NAME \
  --project=PROJECT_ID \
  --zone=ZONE \
  --extensions=ops-agent 
  --inclusion-labels=KEY1=VALUE1,KEY2=VALUE2

Replace the following variables in the command:

  • POLICY_NAME: A name for the VM extension policy.
  • PROJECT_ID: The identifier of the project.
  • ZONE: The target zone; for example, us-central1-f.
  • KEY1=VALUE1,KEY2=VALUE2: A comma-separated list of key-value pairs that define inclusion labels for a selector. VMs must have all the labels specified in a selector (logical AND) to be included. If you specify the --inclusion_labels option multiple times, then the policy targets the VMs that match all the labels in any of the selectors (logical OR). If you omit this option, the policy targets all VMs in the specified zone.

Global

Use the --inclusion-labels option to the gcloud compute global-vm-extension-policies create command to specify a comma-separated list of key-value pairs:

gcloud compute global-vm-extension-policies create POLICY_NAME \
  --project=PROJECT_ID \
  --extensions=ops-agent  \
  --rollout-predefined-plan=slow_rollout \
  --inclusion-labels=KEY1=VALUE1,KEY2=VALUE2

Replace the following variables in the command:

  • POLICY_NAME: A name for the VM extension policy.
  • PROJECT_ID: The identifier of the project.
  • KEY1=VALUE1,KEY2=VALUE2: A comma-separated list of key-value pairs that define inclusion labels for a selector. VMs must have all the labels specified in a selector (logical AND) to be included. If you specify the --inclusion_labels option multiple times, then the policy targets the VMs that match all the labels in any of the selectors (logical OR). If you omit this option, the policy targets all VMs in the specified zone.

Install the Ops Agent with a custom configuration on all VMs

To create a policy that installs the latest version of the Ops Agent on all VMs in a zone or globally, and provides custom configuration for the Ops Agent, do the following:

Console

To create a zonal policy, you can use the Google Cloud console.

  1. In the Google Cloud console, go to the Extension policies page:

    Go to Extension policies

    If you use the search bar to find this page, then select the result whose subheading is Compute Engine.

  2. Click + Create.
  3. In the Name field, enter a name for the policy.
  4. Optional: In the Description field, enter a description for the policy.
  5. In the Priority field, specify a priority number to resolve conflicts between policies. Lower numbers indicate higher priority. The default value is 1000.
  6. In the Zone list, select the zone where you want to apply this policy.
  7. In the Manage extensions section, click Add extension and do the following:
    1. From the Extension list, select Google Cloud's Extension for Ops Agent.
    2. Leave the Version field blank.
    3. In the Config file content field, enter the YAML configuration string for the Ops Agent.
  8. Click Create.

gcloud

To create a zonal or global policy, you can use the gcloud CLI.

Zonal

Use the --config-from-file option to the gcloud compute zone-vm-extension-policies create command to specify a configuration file:

gcloud compute zone-vm-extension-policies create POLICY_NAME \
  --project=PROJECT_ID \
  --zone=ZONE \
  --extensions=ops-agent 
  --config-from-file=ops-agent="OPS_AGENT_CONFIG_PATH"

Replace the following variables in the command:

  • POLICY_NAME: A name for the VM extension policy.
  • PROJECT_ID: The identifier of the project.
  • ZONE: The target zone; for example, us-central1-f.
  • OPS_AGENT_CONFIG_PATH: The path to a file containing the YAML configuration string for the Ops Agent. This file must exist in the environment where you run the gcloud command.

    Configuration YAML for the Ops Agent can be lengthy. Putting the configuration into a file and passing the file to the gcloud CLI is much less error-prone than attempting to enter properly formatted YAML on the command line. Changing this file after creating the policy doesn't update the policy; to update a policy, use the gcloud compute zone-vm-extension-policies update command.

Global

Use the --config-from-file option to the gcloud compute global-vm-extension-policies create command to specify a configuration file:

gcloud compute global-vm-extension-policies create POLICY_NAME \
  --project=PROJECT_ID \
  --extensions=ops-agent  \
  --rollout-predefined-plan=slow_rollout \
  --config-from-file=ops-agent="OPS_AGENT_CONFIG_PATH"

Replace the following variables in the command:

  • POLICY_NAME: A name for the VM extension policy.
  • PROJECT_ID: The identifier of the project.
  • OPS_AGENT_CONFIG_PATH: The path to a file containing the YAML configuration string for the Ops Agent. This file must exist in the environment where you run the gcloud command.

    Configuration YAML for the Ops Agent can be lengthy. Putting the configuration into a file and passing the file to the gcloud CLI is much less error-prone than attempting to enter properly formatted YAML on the command line. Changing this file after creating the policy doesn't update the policy; to update a policy, use the gcloud compute global-vm-extension-policies update command.

If you provide a custom configuration for the Ops Agent when creating or updating an extension policy, then the policy deploys the Ops Agent with the custom configuration. You don't need to manually restart the agent.

The configuration is stored with the extension policy, so don't include sensitive data like passwords in the configuration. Because the configuration is stored with the extension policy, modifying the configuration file does not change the configuration of the agent. You must update the configuration stored with the policy by using the appropriate gcloud command:

When you provide a custom configuration for the Ops Agent, VM Extension Manager copies the configuration to the location used by the Ops Agent for user-specified configuration files on the target VM:

  • Linux: /etc/google-cloud-ops-agent/config.yaml
  • Windows: C:\Program Files\Google\Cloud Operations\Ops Agent\config\config.yaml

If there is a config.yaml file present on the VM, then the extension policy overwrites it.

For general information about using the Google Cloud console and the gcloud CLI to create VM extension policies, see Install VM extensions by using extension policies .

Install the Ops Agent with a fast rollout plan at higher priority (global only)

To create a global policy that installs the latest version of the Ops Agent on by using a higher-than-default priority and the fast-rollout plan, use the following gcloud compute global-vm-extension-policies create command:

gcloud compute global-vm-extension-policies create POLICY_NAME \
  --project=PROJECT_ID \
  --extensions=ops-agent  \
  --rollout-predefined-plan=fast_rollout \
  --priority=500

Replace the following variables in the command:

  • POLICY_NAME: A name for the VM extension policy.
  • PROJECT_ID: The identifier of the project.

Install the Ops Agent on a subset of zones in a Google Cloud project (global only)

To create a global policy that applies to only a subset of zones in a Google Cloud project, you must use a custom rollout plan that specifies the zones in which to apply the policy. For information about creating custom rollout plans, see About rollout plans. To specify a set of zones, use the locationSelector.includedLocations field when creating the rollout plan.

To create an extension policy that uses a custom rollout plan, use the following gcloud compute global-vm-extension-policies create command:

gcloud compute global-vm-extension-policies create POLICY_NAME \
  --project=PROJECT_ID \
  --extensions=ops-agent  \
  --rollout-custom-plan=projects/PROJECT_ID/locations/global/rolloutPlans/CUSTOM_PLAN_NAME

Replace the following variables in the command:

  • POLICY_NAME: A name for the VM extension policy.
  • PROJECT_ID: The identifier of the project.
  • CUSTOM_PLAN_NAME: The name associated with your custom rollout plan.

Verify the installation of the Ops Agent

This section describes how to do the following:

Verify the status or version of an Ops Agent installation

To determine the status or version of the Ops Agent, use one of the following Google Cloud console pages:

Compute Engine

  1. In the Google Cloud console, go to the VM instances page:

    Go to VM instances

    If you use the search bar to find this page, then select the result whose subheading is Compute Engine.

  2. Select a VM from the list.
  3. Click the Observability tab.

Cloud Monitoring

  1. In the Google Cloud console, go to the VM Instances page:

    Go to VM Instances

    If you use the search bar to find this page, then select the result whose subheading is Monitoring.

  2. Select the List view.

When the Ops Agent has been installed on the VM and is collecting logs and metrics, the agent's status is marked with a green checkmark next to the Ops Agent label.

To determine the version of the installed agent, hover over the Ops Agent label on the Compute Engine or Monitoring dashboard.

Verify that the Ops Agent is collecting telemetry

If the Ops Agent has been successfully installed and is running correctly, then it sends metrics to Cloud Monitoring and logs to Cloud Logging.

What's next