You can see the latest product updates for all of Google Cloud on the Google Cloud page, browse and filter all release notes in the Google Cloud console, or programmatically access release notes in BigQuery.
To get the latest product updates delivered to you, add the URL of this page to your feed reader, or add the feed URL directly.
October 09, 2025
Addressed multiple Common Vulnerabilities and Exposures (CVEs) by updating dependencies.
September 18, 2025
Addressed multiple Common Vulnerabilities and Exposures (CVEs) by updating dependencies.
Upgraded the Open Telemetry image from v0.118.0 to v0.119.0 to pick up vulnerability fixes. To understand the changes in each release, review the full changelog for opentelemetry-collector-contrib.
September 02, 2025
Config Sync is now available as part of the standard GKE offering and no longer requires GKE Enterprise. For more details on the removal of GKE Enterprise, see the GKE release notes.
August 21, 2025
Announcing experimental features: help shape the future of Config Sync features by providing direct feedback.
- Introducing PostSync, a feature that lets you run custom actions like cleanup scripts or notifications right after your configurations are synced. We're looking for your feedback to shape its future! Check out the Post Sync discussion to share your thoughts, suggestions, and bug reports before December 1, 2025.
Config Sync now supports syncing from Secure Source Manager git repositories. For more information, see Grant access to Git.
Addressed multiple Common Vulnerabilities and Exposures (CVEs) by updating dependencies.
July 24, 2025
Addressed multiple Common Vulnerabilities and Exposures (CVEs) by updating dependencies.
Updated the git-sync image from v4.3.0 to v4.4.2, which fixes an issue that could cause the git-sync container to crash loop. For more information see git-sync releases.
Fixed a regression introduced in 1.21.0 that occasionally caused Config Sync to become stuck when applying mutation ignored objects.
Fixed an issue where Config Sync waited longer than intended between retry attempts after failing to sync from Helm and OCI sources.
June 26, 2025
Addressed multiple Common Vulnerabilities and Exposures (CVEs) by updating dependencies.
Fixed an issue with the nomos CLI which prevented setting up autocomplete by using the nomos completion command. For more information see Use the nomos command-line tool.
Fixed an issue which prevented a resource conflict metric from being recorded in rare cases.
May 29, 2025
Addressed multiple Common Vulnerabilities and Exposures (CVEs) by updating dependencies.
May 01, 2025
Installing Policy Controller through the ConfigManagement API is no longer supported. For instructions installing Policy Controller, see Installing Policy Controller. For instructions migrating existing Policy Controller instances, see Migrate from the ConfigManagement API to the PolicyController API. This note was added on May 16, 2025.
Fixed an issue preventing ResourceGroup objects from being garbage collected when their corresponding RootSync or RepoSync objects were deleted.
Fixed several issues to improve ResourceGroup status reporting and reliability.
Fixed an issue where drift prevention incorrectly blocked modifications of abandoned resources.
The Config Sync auto-upgrades feature is now unavailable. You can no longer configure auto-upgrade settings and must manually upgrade the Config Sync version. If you currently use auto-upgrades, you must first disable auto-upgrades before you can manually update Config Sync.
Upgraded the Open Telemetry Collector image from v0.103.0 to v0.118.0. This upgrade includes a breaking change where the default OTLP component endpoint is now localhost instead of 0.0.0.0. You will be impacted only if you use a customized configuration for the built-in Otel Collector within Config Sync, and you can explicitly specify 0.0.0.0 for endpoints to ensure that your monitoring solution continues to function correctly. To understand the changes in each release, review the full changelog for opentelemetry-collector-contrib.
The nomos vet command now supports a --threshold flag to proactively validate the number of objects in your Config Sync repository. You can use this flag in validation pipelines to prevent sync failures caused by exceeding the underlying etcd size limits when your repository contains a large number of objects. For more information, see Enforce the maximum number of objects to sync.
Deleting a RootSync or RepoSync now removes its management metadata from all managed objects. This allows objects to be adopted by their new managers, simplifying the procedure for splitting a large configuration repository across multiple RootSync or RepoSync objects. For more information, see Break up a repository into multiple repositories.
Addressed multiple Common Vulnerabilities and Exposures (CVEs) by updating dependencies.
Fixed an issue impacting the Ignore object mutations feature. The client.lifecycle.config.k8s.io/mutation: ignore annotation was not always effective, causing Config Sync to potentially overwrite changes made directly to annotated resources in the cluster. Config Sync now correctly ignores mutations on these resources.
March 27, 2025
Addressed multiple Common Vulnerabilities and Exposures (CVEs) by updating dependencies.
March 06, 2025
Fixed an issue where ConfigManagement uninstall could get stuck when Policy Controller was enabled via ConfigManagement. This was caused by Policy Controller finalizers not being properly removed during the uninstallation process.
January 30, 2025
Fixed an issue that was causing Container Registry and Artifact Registry authentication tokens to expire before being refreshed. For more information, see known issue Unable to generate access token for OCI source.
Fixed an issue that incorrectly reported managed resources as "Not Found" when an API Service backend became unhealthy. For more information, see known issue API discovery errors can cause managed objects to incorrectly be marked as "Not Found".
December 05, 2024
Hierarchy Controller is no longer available to install. If Hierarchy Controller is still configured, Config Sync upgrades are blocked. To upgrade Config Sync, disable Hierarchy Controller. This release note was updated on May 9, 2025. The previous note suggested Hierarchical Namespace Controller (HNC) as a migration option, but HNC is now archived.
Improved the manual installation process for Config Sync. When you install Config Sync manually using kubectl (not recommended), Config Sync is now deployed directly without relying on the Config Management Operator. This change results in simplified architecture and reduced resource use on your cluster. If you manually installed Config Sync using kubectl, follow the guide to uninstall the Config Management Operator before you upgrade.
When you use Config Sync to manage configurations that are stored in OCI repositories (such as Artifact Registry), you can now enhance your security posture with custom signature verification. Config Sync integrates with your existing signature verification server deployed as a Kubernetes admission webhook, which helps ensure only trusted OCI images are used in your deployments. See the Sync OCI artifacts guide for setup instructions.
Introduced a new field for stopping and resuming syncing. This field is available on clusters with Config Sync auto-upgrades or with Config Sync version 1.20.0. The new field makes it easier to pause syncing by setting the spec.configSync.stopSyncing field to true.
To optimize resource use, Config Sync installations managed through Fleet no longer include the ConfigManagement Operator or the ConfigManagement CRD. These components are automatically removed when you upgrade to version 1.20.0 or later. This change reduces Config Sync's resource consumption in your cluster. See Config Sync architecture for details.
Upgraded the git-sync dependency from v4.2.4 to v4.3.0 to pick up a fix for lingering Git lock files and other vulnerability fixes.
Fixed a bug that prevented the applyset.kubernetes.io/part-of label from being correctly removed from managed objects when they were no longer managed by Config Sync. This fix improves the accuracy of label information.
Fixed an issue that could cause sync delays due to retry backoff problems. This fix helps ensure more timely and consistent updates to your clusters.
October 17, 2024
Improved the security of the git-sync container by upgrading the base image to address known vulnerabilities.
September 26, 2024
You can now configure Config Sync fleet defaults with gcloud commands starting in gcloud version 494.0.0. See the Config Sync gcloud documentation for reference.
Config Sync now supports GitHub App authentication for GitHub repositories. See Grant access to Git for more information. This release note was added on October 4, 2024.
Upgraded the git-sync dependency from v4.2.3 to v4.2.4.
August 29, 2024
Terraform version 5.41.0 introduced a new field to the google_gke_hub_feature_membership: config_sync.enabled. Because the default value of this field is false, it causes Config Sync installations to fail when Terraform is upgraded to version 5.41.0. For more information, including workarounds, see the known issue entry. This issue affects all supported Config Sync versions. This note was added on September 11, 2024.
The spec.git and spec.enableLegacyFields fields of the ConfigManagement object have been removed. The spec.enableMultiRepo field is now set to true by default, automatically enabling the RootSync API. RootSync provides the same core functionality, along with additional features.
If you currently configure Git settings within a ConfigManagement object, to avoid disruptions, before upgrading you must migrate this configuration to a RootSync object.
Hierarchy Controller will not be available after December, 2024. After December, 2024, you can't install Hierarchy Controller and Config Sync will be blocked from upgrades if Hierarchy Controller is configured as a configmanagement fleet feature or through the ConfigManagement API. To continue using similar functionality, migrate from Hierarchy Controller to Hierarchical Namespace Controller.
This note was added on September 6, 2024 and edited on September 9, 2024 for clarification.
Optimized Config Sync resource usage by implementing watch filtering with ApplySet ("applyset.kubernetes.io/" labels and annotations). This reduces reconciler Deployment memory consumption by limiting events and cached objects to those relevant to the managed package. For more information on the resource usage optimization, see Config Sync Watch Filtering v1.18 vs v1.19.
Config Sync now enables loading files from directories beyond the Kustomize root during rendering. For more information, refer to Configure Kubernetes with Kustomize.
Improved support for private registries. If you've configured a private registry for your cluster, Config Sync now automatically detects and updates the image references within its reconciler Deployments to point to the corresponding images in your private registry.
Upgraded bundled Helm version from v3.14.4 to v3.15.3 to pick up vulnerability fixes. To understand the changes in each release, review the changelogs.
Upgraded the Open Telemetry image from 0.102.0 to 0.103.0 to pick up vulnerability fixes. To understand the changes in each release, review the full changelog for opentelemetry-collector-contrib.
Fixed some inaccuracies in status updates and metrics reports.
July 25, 2024
Improved error handling in the oci-sync container by adding exponential backoff.
Upgraded bundled Kustomize version from v5.3.0 to v5.4.2 to pick up vulnerability fixes. To understand the changes in each release, review the changelogs.
Fixed a regression introduced in 1.17.0 that caused Config Sync to crash when connecting to certain Kubernetes clusters. GKE clusters were not affected by this issue. For more details, please refer to Fixed: Config Sync reconciler is crashlooping.
Fixed a regression in 1.17.3 causing SSH authentication failures with GitHub. This was resolved by upgrading the git-sync dependency from v4.2.1 to v4.2.3. For more details, please refer to Fixed: Git SSH Authentication Failure with GitHub.
June 27, 2024
Reverted an undocumented change to a metric name. The Cloud Monitoring metric current_declared_resources (introduced in version 1.16.1) has been renamed to its original name, declared_resources. For reference see Monitor Config Sync with Cloud Monitoring.
Upgraded the Open Telemetry image from v0.99.0 to v0.102.0 to pick up vulnerability fixes. To understand the changes in each release, review the full changelog for opentelemetry-collector-contrib.
Resolved an issue that prevented the declared_resources metric from decrementing when an object became unmanaged by Config Sync.
May 30, 2024
Upgraded bundled Helm version from v3.14.3 to v3.14.4 to pick up vulnerability fixes. To understand the changes in each release, review the changelogs.
Upgraded the Open Telemetry image from v0.91.0-gke.9 to v0.99.0-gke.1 to pick up vulnerability fixes. To understand the changes in each release, review the full changelog for opentelemetry-collector-contrib.
Fixed an issue where Config Sync installation would fail when using a private registry with a specified port in the image URL.
May 02, 2024
Manually installing Policy Controller through the ConfigManagement object is no longer supported. Instead, use Policy Controller Google Cloud CLI commands to install and configure Policy Controller. This note was added on July 2, 2024.
When syncing from Helm, Config Sync now retries faster on errors with exponential backoff.
Reduced memory footprint in reconcilers by not loading the OpenAPI when the Config Sync admission webhook is disabled.
On Autopilot clusters, the helm-sync container CPU request is changed from 150m to 250m, and memory request is changed from 256Mi to 384Mi. For information on resource requirements, see Resource requests.
Upgraded bundled Helm version from v3.13.3 to v3.14.3 to pick up vulnerability fixes. To understand the changes in each release, review the changelogs.
Policy Controller now has its own release notes page. For future announcements, visit Policy Controller release notes.
Mitigated an issue periodically occurring that could cause an error when the authentication token expires for Cloud Source Repositories, prompted by a race condition between the gcenode-askpass-sidecar container validating the credentials and the git-sync container trying to use those credentials. This note was added on June 18, 2024.
Installing Policy Controller 1.18.0 or newer will fail unless you first enable the anthospolicycontroller.googleapis.com API. For more information on directly installing and managing Policy Controller, see Install Policy Controller.
Dynamic namespace selection using the spec.mode field in the NamespaceSelector CRD is now generally available (GA). This feature supports deploying namespace-scoped resources in matching Namespaces statically-declared in the source of truth and dynamically present on the cluster. For more information, refer to NamespaceSelector mode.
Config Sync now supports specifying CA certificates for helm and OCI source types. This is surfaced on the caCertSecretRef field on the RootSync and RepoSync APIs. For more information, refer to RootSync and RepoSync fields.
Policy Controller bundles have been updated to the following versions: cis-gke-v1.5.0: 202403.0, nist-sp-800-190: 202403.0, nist-sp-800-53-r5: 202403.0, pci-dss-v3.2.1: 202403.0, pci-dss-v4.0: 202403.0, policy-essentials-v2022: 202403.0, pss-baseline-v2022: 202403.1, pss-restricted-v2022: 202403.1. For reference, see Policy Controller bundles overview.
March 21, 2024
The constraint template library includes a new template: K8sPSSRunAsNonRoot. For reference, see the Constraint template library.
Policy Controller bundles have been updated to the following versions: cis-gke-v1.4.0: 202402.0-preview, nist-sp-800-190: 202402.0, nist-sp-800-53-r5: 202402.0, pci-dss-v3.2.1: 202402.0, pss-baseline-v2022: 202402.0, pss-restricted-v2022: 202402.0. For reference, see Policy Controller bundles overview.
Fixed a regression introduced in 1.16.0 that limits the length of the Secret name referenced in the spec.git.secretRef.name field of the RootSync object.
Fixed a regression introduced in 1.17.0 that caused Config Sync to sometimes fail to pull the latest commit from a Git branch by upgrading git-sync (Config Sync dependency for pulling from git) from v4.1.0 to v4.2.1.
February 22, 2024
The constraint template library includes a new template: K8sRestrictAdmissionController. For reference, see the Constraint template library.
The constraint template library includes a new template: K8sCronJobAllowedRepos. For reference, see the Constraint template library.
Added the authentication type k8sserviceaccount for syncing from OCI images and Helm charts hosted in Artifact Registry. For more details, see Grant Config Sync read-only access to OCI and Grant Config Sync read-only access to Helm.
Simplified the steps to export metrics to Cloud Monitoring. For more details, see Configure Cloud Monitoring with Workload Identity.
Fixed the unrecognized label error in the otel-collector configuration that caused kustomize metrics to be rejected.
February 16, 2024
Upgraded git-sync (Config Sync dependency for pulling from git) from v3.6.9 to v4.1.0 to pick up enhancements, such as improved efficiency and race condition fixes. This contains a breaking change that short commit SHA is no longer accepted in the spec.git.revision field of RootSync and RepoSync. If you want to sync from a Git commit, use a full commit SHA in the spec.git.revision field. For more details, please refer to Configuration for the Git repository. This release note was updated February 16, 2024 with a correction to the version number.
January 25, 2024
The constraint template library includes a new template: K8sDisallowInteractiveTTY. For reference, see the Constraint template library.
Policy Controller bundles have been updated to the following versions: cis-k8s-v1.5.1: 202312.1, cost-reliability-v2023: 202312.0, nist-sp-800-190: 202312.1, nist-sp-800-53-r5: 202312.1, nsa-cisa-k8s-v1.2: 202312.1, pci-dss-v3.2.1: 202312.1, psp-v2022: 202312.0. For reference, see Policy Controller bundles overview.
Upgraded bundled Helm version from v3.13.1 to v3.13.3 to pick up vulnerability fixes. To understand the changes in each release, review the changelogs.
Upgraded bundled Kustomize version from v5.1.1 to v5.3.0 to pick up vulnerability fixes. To understand the changes in each release, review the changelogs.
Fixed a race condition that could cause deadlock when uninstalling Config Sync. This issue could sometimes occur if a RepoSync was managed by a RootSync with foreground deletion propagation enabled.
Fixed an issue that could cause RootSyncs and RepoSyncs to be reported as Current before they had been reconciled.
Fixed an issue where the log level could not be set for the otel-agent container.
Fixed an issue where RepoSync Secrets were not garbage collected when switching between Secret refs or types.
Fixed a performance issue where the config-management-operator was continuously updating the webhook definition, causing unnecessary churn in the apiserver and etcd.
December 11, 2023
Added validation to block declaration in the source of truth of the Config Sync controller namespaces, including resource-group-system and config-management-monitoring namespaces. For reference see KNV1034: IllegalNamespaceError. This note was updated on January 10, 2024
The nomos command-line tool now requires the kubectl plugin gke-gcloud-auth-plugin to be installed to interact with GKE, and your KUBECONFIG must be configured to use it by calling gcloud container clusters get-credentials. Versions of the gcloud CLI older than 430.0.0 also require setting USE_GKE_GCLOUD_AUTH_PLUGIN=True. For reference, see Prerequisites for the nomos command-line tool.
Upgraded git-sync (Config Sync dependency for pulling from git) from v3.6.9 to v4.0.0 to pick up enhancements, such as improved efficiency and race condition fixes. This contains a breaking change that short commit SHA is no longer accepted in the spec.git.revision field of RootSync and RepoSync. If you want to sync from a Git commit, use a full commit SHA in the spec.git.revision field. For more details, please refer to Configuration for the Git repository. This release note is incorrect; see entry for February 16, 2024.
Fixed an issue with the nomos CLI not authenticating correctly when run inside a Kubernetes Pod. Now, to authenticate to another cluster while inside a pod, you must explicitly set KUBECONFIG to point to the kubectl config, otherwise the downward API is used to authenticate with the local cluster.
Added a new field spec.mode in the NamespaceSelector CRD as a preview feature to support selecting namespace-scoped resources matching both statically-declared Namespaces in the source of truth and those dynamically present on the cluster. For more details, please refer to Limit which namespaces a config affects.
Added a new field spec.override.namespaceStrategy in the RootSync API to control whether Namespaces should be created implicitly or not if Namespace configs are missing from the source of truth. For reference see RootSync and RepoSync fields.
Added a new field spec.override.roleRefs in the RootSync API to customize root reconciler permissions other than cluster-admin. For reference see RootSync and RepoSync fields.
Added known_hosts support for Git connection over SSH in RootSync and RepoSync APIs. For more details on SSH key pair see Install Config Sync.
The constraint template library includes a new template: K8sRequireAdmissionController. For reference, see the Constraint template library.
Policy Controller bundles have been updated to the following versions: asm-policy-v0.0.1: 202311.0, cis-k8s-v1.5.1: 202311.0, cost-reliability-v2023: 202311.0, nist-sp-800-190: 202311.0, nist-sp-800-53-r5: 202311.0, nsa-cisa-k8s-v1.2: 202311.0, pci-dss-v3.2.1: 202311.0, policy-essentials-v2022: 202311.0, psp-v2022: 202311.0, pss-baseline-v2022: 202311.0, pss-restricted-v2022: 202311.0. For reference, see Policy Controller bundles overview.
The constraint template library's K8sNoExternalServices template now supports the "networking.gke.io/load-balancer-type": "Internal" annotation. For reference, see Constraint template library.
Reduced Config Sync reconciler default CPU and memory requests on GKE Standard clusters. Increased Config Sync reconciler default CPU and memory requests to avoid throttling and reduce time to sync by up to 25%, and increased default limits on GKE Autopilot clusters to avoid out of memory errors for most workloads. For reference see Resource requests.
November 13, 2023
Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: a1f01f4 ).
Policy Controller bundles have been updated to the following versions: asm-policy-v0.0.1: 202310.0, cis-k8s-v1.5.1: 202310.0, cost-reliability-v2023: 202310.0-preview, nist-sp-800-190: 202310.0, nist-sp-800-53-r5: 202310.0, nsa-cisa-k8s-v1.2: 202310.0, pci-dss-v3.2.1: 202310.0, policy-essentials-v2022: 202310.0, psp-v2022: 202310.0, pss-baseline-v2022: 202310.0, pss-restricted-v2022: 202310.0. For reference, see Policy Controller bundles overview.
The constraint template library's K8sPSPAllowedUsers, K8sPSPAllowPrivilegeEscalationContainer, K8sPSPAutomountServiceAccountTokenPod, K8sPSPCapabilities, K8sPSPFlexVolumes, K8sPSPForbiddenSysctls, K8sPSPFSGroup, K8sPSPHostFilesystem, K8sPSPHostNamespace, K8sPSPHostNetworkingPorts, K8sPSPPrivilegedContainer, K8sPSPProcMount, K8sPSPReadOnlyRootFilesystem, K8sPSPSELinuxV2, K8sPSPVolumeTypes, and K8sRequiredProbes no longer raise violations during updates of existing objects for immutable fields.
Updated the Open Telemetry image from 0.86.0 to 0.87.0 to address security vulnerabilities. For more information about these changes, see the full changelog for opentelemetry-collector-contrib.
October 19, 2023
Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 3e66ee2).
The constraint template library includes a new template: K8sAvoidUseOfSystemMastersGroup. For reference, see the Constraint template library.
The constraint template library includes a new template: K8sPSPWindowsHostProcess. For reference, see the Constraint template library.
Policy Controller bundles have been updated to the following versions: asm-policy-v0.0.1: 202309.0, cis-k8s-v1.5.1: 202309.0, cost-reliability-v2023: 202309.0, nist-sp-800-190: 202309.0, nist-sp-800-53-r5: 202309.0, nsa-cisa-k8s-v1.2: 202309.0, pci-dss-v3.2.1: 202309.0, policy-essentials-v2022: 202309.0, psp-v2022: 202309.0, pss-baseline-v2022: 202309.0, pss-restricted-v2022: 202309.0. For reference, see Policy Controller bundles overview.
Updated the Open Telemetry image from 0.54.0 to 0.86.0 to address security vulnerabilities. otelcontribcol:v0.86.0 contains breaking changes. For more information about these changes, see the full changelog for opentelemetry-collector-contrib.
Fixed a recurring transient error in the RootSync and RepoSync API. Transient errors are retried internally and surfaced to RootSync and RepoSync if failed eventually.
September 21, 2023
Supported configuring the period that Config Sync waits before re-pulling the latest chart with the field spec.helm.period in RootSync or RepoSync. The default is 1 hour. For more information, see RootSync and RepoSync fields.
Fixed an issue related to metric labels commit and type that could cause an increase in Google Cloud Monitoring metric write throughput. The issue was resolved by preventing high-cardinality labels commit and type from being sent to Google Cloud Monitoring. The two labels were introduced in Anthos Config Management version 1.15.0. For information on the labels, see Config Sync Metric Labels.
Fixed a repeating error in the otel-collector deployment log that has been occuring since Anthos Config Management version 1.15.0.
The constraint template library's K8sRestrictRbacSubjects template now returns all violation messages for ClusterRoleBinding or RoleBinding with more than one violation. For reference, see Constraint template library.
August 21, 2023
Added a new configsync.gke.io/deletion-propagation-policy annotation for use on RootSync and RepoSync, for configuring foreground cascading deletion as a preview feature. When set to Foreground, the resource objects managed by it will be deleted when the RootSync or RepoSync is deleted. To learn more, see Bulk delete objects.
The Helm chart version field spec.helm.version in RootSync and RepoSync can now be specified as a range of values from which Config Sync pulls the latest values. For reference see Configuration for the Helm repository.
Added the spec.override.logLevels field to RootSync and RepoSync, for configuring the log level of the containers of reconciler Pods. For more information, see RootSync and RepoSync fields and General best practices.
The constraint template library's K8sRequireDaemonsets template now supports restricting the use of NodeSelector in required Daemonset using the new restrictNodeSelector parameter. For reference, see Constraint template library.
Policy Controller bundles have been updated to the following versions: cis-k8s-v1.5.1: 202307.1, pci-dss-v3.2.1: 202307.0, policy-essentials-v2022: 202307.1, pss-baseline-v2022: 202307.0, pss-restricted-v2022: 202307.0. For reference, see Policy Controller bundles overview.
Upgraded bundled Kustomize version from v5.1.0 to v5.1.1 to pick up vulnerability fixes. To understand the changes in each release, review the changelogs.
Reduced the resource requirements for the reconciler Pod when no Kustomize rendering is needed. For information on resource requirements, see Resource requests.
Increased the default timeout for Kubernetes API requests from 5 seconds to 15 seconds. This allows more requests to succeed with fewer retries when the control plane is under load, and reduces the need to override the timeout with spec.override.apiServerTimeout in RootSync and RepoSync. To learn more, see Configuration for overriding the resource requests and limits of a root or namespace reconciler.
Fixed an issue causing Config Sync to prematurely stop tracking managed resource objects when pruning them before they are deleted from the cluster. Those objects will now appear in the Cloud Console and nomos status until the deletion has reconciled.
Fixed an issue preventing Config Sync from pruning managed resources correctly when the apiGroup of those resources is renamed in the source of truth.
Known Issue for Policy Controller: If you're using ExpansionTemplates and mutators, Policy Controller will not apply the mutators for the generated resources.
There is a memory issue in the reconciler due to the discovery client loading the OpenAPI for schema validations. The issue is only observed on Autopilot clusters with many CRDs installed. You can increase the memory request with the override.resources API. To learn more, see Configuration for overriding the resource requests and limits of a root or namespace reconciler.
Added the spec.configSync.metricsGcpServiceAccountEmail field to the gcloud apply spec fields to remove 2 manual steps required to export Config Sync metrics from a cluster with Workload Identity enabled to Cloud Monitoring. This note was updated on September 7, 2023.
The constraint template library's GkeSpotVMTerminationGrace template adds the includePodOnSpotNodes parameter, which requires enabling referential constraints. For reference, see Constraint template library.
The Config Sync feature to sync directly from Helm repositories (including OCI-based ones) is generally available (GA). Google Artifact Registry is the preferred Helm registry. To learn more, see Sync Helm charts from Artifact Registry and Configuration for the Helm repository.
Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 616aa8f).
ExpansionTemplates APIs promoted to v1beta1.
Policy Controller add new bundles: nist-sp-800-190, nist-sp-800-53-r5, and nsa-cisa-k8s-v1.2. For reference, see Policy Controller bundles overview.
Added a new field spec.helm.valuesFileRefs in RootSync and RepoSync to allow specifying Helm values files in ConfigMaps. For reference, see Configuration for the Helm repository.
July 27, 2023
Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 6aad7b6).
The constraint template library includes a new template: GkeSpotVMTerminationGrace. For reference, see the Constraint template library.
The constraint template library includes a new template: K8sPodResourcesBestPractices. For reference, see the Constraint template library.
The constraint template library has been updated to use the templates.gatekeeper.sh/v1 API, which includes strict validation of parameter fields, for all templates. For a list of template parameter fields, see the Constraint template library.
Upgraded bundled Kustomize version from v5.0.3 to v5.1.0 to pick up vulnerability fixes. To understand the changes in each release, review the changelogs.
Upgraded bundled Helm version from v3.11.3 to v3.12.2 to pick up vulnerability fixes. To understand the changes in each release, review the changelogs.
June 28, 2023
The constraint template library includes a new template: K8sRequireBinAuthZ. For reference, see the Constraint template library.
The constraint template library includes a new template: K8sRestrictAutomountServiceAccountTokens. For reference, see the Constraint template library.
The constraint template library includes a new template: K8sRestrictRoleRules. For reference, see the Constraint template library.
Fixed a formatting issue in nomos status --name. For more information, see nomos status flags.
June 06, 2023
The Anthos Config Management SKU and Anthos Policy Controller SKU are end-of-sale. As of September 4, 2023, you must have an Anthos license to use Anthos Config Management. If you are using the Anthos Config Management SKU, migrate to an Anthos license by enabling the Anthos API on your project.
May 25, 2023
Added a new field spec.helm.deployNamespace in the RootSync API to support specifying which namespace to deploy the rendered chart. For more information, see RootSync and RepoSync fields.
The constraint template library includes a new template: K8sHorizontalPodAutoscaler. For reference, see the Constraint template library.
The constraint template library's K8sStorageClass template now supports an allowed list of storage classes using the new allowedStorageClasses parameter. For reference, see Constraint template library.
Upgraded bundled Kustomize version from v5.0.1 to v5.0.3 to pick up vulnerability fixes. To understand the changes in each release, review the changelogs.
The constraint template library's K8sReplicaLimits template now includes the object's actual kind instead of always deployment in violation messages. For reference, see Constraint template library.
May 05, 2023
The spec.git fields of the ConfigManagement object are deprecated and are scheduled for shut down on or after May 15, 2024. The
RootSync API offers the
same functionality and other benefits like syncing from multiple sources and improved
observability. For information about the benefits and instructions for migrating
your Git configuration to the RootSync object, see
Migrate your ConfigManagement object.
May 04, 2023
Depending on the frequency of production repository commits and variance in Kubernetes Kinds being deployed, the two new metric labels commit and type introduced in this release can cause an increase in Google Cloud Monitoring metric write throughput. This can result in an increase of monthly billing. The affected metrics are apply_duration_seconds, api_duration_seconds, declared_resources, apply_operations_total. To avoid an increase in billing or if these metrics are not needed, follow the instructions to patch the otel collector deployment with ConfigMap and remove the affected metrics from the filter/cloudmonitoring allowlist. This note was updated on August 7, 2023.
Upgraded bundled Kustomize version from v4.5.2 to v5.0.1. Config Sync leverages the Kustomize executable to render the configurations under the hood. To understand the changes in each release, review the changelogs. Kustomize v5.0.0 contains breaking changes. For more information about these changes, see the full changelog for Kustomize v5.0.0. This note was updated on May 18, 2023.
The constraint template library's K8sEnforceConfigManagement template adds new requireDriftPrevention and requireRootSync parameters, which requires enabling referential constraints. For reference, see Constraint template library.
The constraint template library includes a new template: K8sContainerEphemeralStorageLimit. For reference, see the Constraint template library.
The constraint template library includes a new template: K8sDisallowedRepos. For reference, see the Constraint template library.
The constraint template library includes a new template: K8sRestrictNfsUrls. For reference, see the Constraint template library.
Added new metric labels: commit and type. These tags make it easier to detect when an error has been resolved. If you have a custom otel-collector ConfigMap, you should update it to filter out these tags for the Kubernetes exporter. For more information, see Config Sync Metric Labels.
Added a --name flag to nomos status to support filtering status by RootSync or RepoSync names. For more information, see nomos status flags
Changed error message ResourceFightWarning to ResourceFightError so that resource fighting conflict can be exposed as errors in nomos status and RootSync/RepoSync status.
Upgraded bundled Helm version from v3.6.3 to v3.11.3. Config Sync leverages the Helm executable to render the configurations under the hood. For more information, see the changelog for Helm v3.11.3. This note was updated on May 22, 2023.
Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: effa347).
March 23, 2023
Alpha release of AssignImage mutator, which allows mutation of Docker image paths. For reference, see AssignImage under Mutation in the OPA Gatekeeper documentation.
The constraint template library includes a new template: VerifyDeprecatedAPI. For reference, see the Constraint template library.
The constraint template library's K8sPodsRequireSecurityContext template now supports an exempt-list of Images using the new exemptImages parameter. For reference, see Constraint template library.
The constraint template library's K8sRequireCosNodeImage template now supports an exempt-list of OS images using the new exemptOsImages parameter. For reference, see Constraint template library.
Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 8170c5f).
Stopped exposing the "unable to load /repo/source/error.json" transient error in the RootSync and RepoSync API.
Config Sync disallows RootSync and RepoSync to do self management and a KNV1069 SelfManageError will be reported. This note was updated on April 12, 2023 and on May 11, 2023.
Fixed an issue where users are seeing an inaccurate PENDING instead of INSTALLED state on Policy Controller status. This change is added for gcloud version 427.0.0. This note was updated on April 19, 2023.
Fixed an issue in the nomos CLI so that it works for standalone Config Sync.
Fixed an issue causing a Kubernetes Service object not syncing without the .spec.ports field being specified.
Fixed an issue of accidental deletion of resources caused by a race condition between *-sync, hydration-controller and reconciler containers.
February 23, 2023
Increased the helm-sync container CPU request to 50m. For information on resource requirements, see Resource requests.
Updated the spec.override.resources field on RootSync and RepoSync objects to let you override the default resource amounts (for example, CPU or memory) requested by the helm-sync container.
January 26, 2023
The constraint template library's K8sPSPForbiddenSysctls template now supports an allow-list of sysctls using the new allowedSysctls parameter. For reference, see Constraint template library.
Config Sync now includes resource-related metrics labels in Google Cloud Monitoring. These labels were previously added to the Prometheus monitoring pipeline in Config Sync version 1.14.0. The labels are available under the "Group By" filter options in the Google Cloud Console. For more information on metrics, see Monitoring Config Sync.
Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: c61db24).
Fixed an issue where the nomos image did not contain the nomos CLI.
December 08, 2022
Config Sync now ignores validating and applying any resource configuration that has the annotation config.kubernetes.io/local-config with any value except for "false", instead of ignoring only when the value is "true". This is consistent with its behavior in kpt.
The following five metrics are removed because these metrics aren't needed for monitoring system performance or health:
rendering_countskip_rendering_countresource_override_countgit_sync_depth_override_countno_ssl_verify_count
For information on current metrics, see Monitor Config Sync.
The constraint template library includes a new template: K8sRequireValidRangesForNetworks. For reference see Constraint template library.
The constraint template library includes a new template: K8sRestrictRbacSubjects. For reference see Constraint template library.
The following enhancements are made to Config Sync metrics:
- Enhanced the histogram distribution bounds for the
parser_duration_secondsandapply_duration_secondsmetrics to support longer durations. - Enhanced the
last_sync_timestampmetric to prevent timeseries with empty commits. - Added a new label called
controllerinto theapply_operationsmetric to track whether the operation is from the applier or the remediator. - Support the
errorclasslabel of thereconciler_errorsmetric correctly.
For more details, see Monitor Config Sync.
Added resource tags to all Config Sync metrics to identify the source component. For more information, see Config Sync Metric Tags.
Fixed a known compatibility issue in Config Sync that was announced in Anthos Config Management 1.13.1 affecting Autopilot on GKE 1.23 and later. Config Sync is now compatible with Autopilot clusters on all supported GKE versions.
Various reliability and stability improvements to Config Sync.
Added a field spec.override.apiServerTimeout in RootSync and RepoSync, for configuring the timeout for requests to the API server.
The first edition of the Config Sync Service Level Indicators (SLIs) is published. You can set up alerts based on these SLIs, and get alerts if a threshold is hit. To learn more about the Config Sync SLIs, see Use Config Sync SLIs.
The constraint template library includes a new template: K8sBlockAllIngress. For reference see Constraint template library.
The constraint template library includes a new template: K8sBlockCreationWithDefaultServiceAccount. For reference see Constraint template library.
The constraint template library includes a new template: K8sBlockObjectsOfType. For reference see Constraint template library.
The constraint template library includes a new template: K8sEnforceCloudArmorBackendConfig. For reference see Constraint template library.
The constraint template library includes a new template: K8sEnforceConfigManagement. For reference see Constraint template library.
The constraint template library includes a new template: K8sRequireDaemonsets. For reference see Constraint template library.
The constraint template library includes a new template: K8sRequireDefaultDenyEgressPolicy. For reference see Constraint template library.
October 27, 2022
Changed the default Helm release namespace from config-management-system to default, if spec.helm.namespace isn't specified. Note that the value specified in spec.helm.namespace is only used as the value of Release.Namespace declared in your Helm template; otherwise, the namespace default will be used.
Added the spec.helm.values field in RootSync and RepoSync to allow overriding the default values that accompany the Helm chart.
The constraint template library includes a new template: K8sBlockLoadBalancer. For reference, see Constraint template library.
The constraint template library's K8sHttpsOnly template now supports Ingress blocks which do not include tls: using the new tlsOptional: true parameter. For reference, see Constraint template library.
Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 600a68d).
Config Sync now handles exporting metrics correctly with the right permissions and resource names after the update to Open Telemetry v0.54.0 which was introduced in ACM 1.12.2.
Fixed a Prometheus exporter error in the otel-collector by resolving a discrepancy between components regarding the description of the pipeline_error_observed metric.
GKE version 1.23 introduced a change that made 1.23 Autopilot clusters incompatible with Config Sync. To work around this issue, use Config Sync on an Autopilot cluster with version 1.22 or earlier. This note was updated on November 4, 2022.
The 409.0.0 Google Cloud CLI release introduced two bugs for Anthos Config Management. The version command incorrectly prints "NA" instead of the current version. The status command prints an incorrect message about unreachable regions. This note was updated on November 10, 2022.
September 15, 2022
On October 31, 2022, we will introduce a new billing SKU for Policy Controller which is a part of Anthos Config Management. However, we won't make any changes to the pricing or billing model.
Customers who use Policy Controller do not need to take any action.
Non-Anthos customers will see a new SKU appear on their monthly bill, but there is no change in pricing. The SKU "Anthos Configuration Management" (81D5-A275-98BF) will be replaced with the SKU "Anthos Policy Controller" (6707-0251-B8E2).
Anthos customers will not see a change as their usage of Policy Controller is covered by their Anthos license.
Anthos Config Management release 1.10 is no longer supported with the release of Anthos Config Management 1.13. For reference, see Get support.
Increased the resource-group-controller container memory request to 200Mi to avoid OOMKilled on Autopilot clusters.
Stopped the nomos command line tool from panicing when nomos status is unable to fetch ResourceGroups.
Fixed an issue that could cause accidental pruning when API resource discovery requests failed.
Added optimizations to the reconciler and reconciler-manager to reduce API requests by avoiding unnecessary resource object updates.
Config Sync now handles empty initContainers fields in managed configs correctly.
The Config Sync feature to sync configurations stored as OCI images in Google Artifact Registry or Container Registry is generally available (GA). To learn more, see Sync OCI artifacts from Artifact Registry.
Config Sync is open sourced. We are open to contributions and bug fixes if you want to get involved in development of Config Sync. You can also use the repository to track ongoing work, or build from source to try out bleeding-edge functionalities.
Config Sync supports syncing from private Helm repositories (including OCI-based ones) as a preview feature. Google Artifact Registry is the preferred Helm registry. To learn more, see Sync Helm charts from Artifact Registry.
The Google Cloud Console now shows the sync status for all syncs in clusters registered to fleet. It also allows for drilling down to see the reconciliation status for individual resources. To learn more, see View Config Sync status in the Google Cloud console.
Config Sync now supports user-provided CA certificates for verifying HTTPS connections to Git servers. To learn more, see Configuration of the Git Repository.
The constraint template library includes a new template: K8sStorageClass. Requires Anthos Config Management version 1.12.1 or higher. For reference see Constraint template library.
The contraint template library's K8sEmptyDirHasSizeLimit template now supports regular expression matching of exempt volume names by using the new exemptVolumesRegex parameter. For reference see Constraint template library.
The contraint template library's K8sMemoryRequestEqualsLimit template now supports regular expression matching of exempt container names by using the new exemptContainersRegex parameter. For reference see Constraint template library.
August 18, 2022
Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 8f1ef8c).
Updated the built-in Open Telemetry image to v0.54.0 to include a bug fix for using ADC with Cloud Spanner receiver.
Fixed the reconciler Pod CrashLoopBackoff issue caused by the git-sync container starting before the gcenode-askpass-sidecar.
Added cluster-autoscaler.kubernetes.io/safe-to-evict: "true" annotation to the reconciler Pod so that it does not block Cluster Autoscaler scale down.
July 21, 2022
The constraint template library includes a new template: K8sRequireCosNodeImage. For reference, see Constraint template library.
Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: c370036).
Fixed the resource name length validation issue caused by long RepoSync names or long namespace names. The new maximum length is 253 characters, instead of 63 characters.
June 30, 2022
Shell access is disabled by default in the Config Sync hydration-controller container. This disables the ability to use Kustomize remote bases. To use Kustomize remote bases, enable shell access by setting the field spec.override.enableShellInRendering: true in RootSync and RepoSync.
Policy Controller now supports Cloud Monitoring. It will automatically export runtime metrics for both Cloud Monitoring and Prometheus. Users can also configure which monitoring backends metrics are exported to. To learn more, see Monitor Policy Controller.
Config Sync removed resource limits from reconciler-manager, reconciler, and git-importer to make them burstable.
Config Sync increased resource limits of admission-webhook (cpu: 1, memory: 2Gi) and otel-agent (cpu: 1, memory: 1Gi).
Anthos Config Management is now compatible with GKE Autopilot with some cluster requirements. Policy Controller mutations are not compatible with Autopilot. Config Sync resource requests and limits adjustments will be further adjusted by GKE Autopilot. To learn more, see Install Config Sync.
Config Sync supports syncing configurations stored as OCI images in Google Artifact Registry or Container Registry as a preview feature. To learn more, see Publish config images to Artifact Registry.
Added a field spec.override.reconcileTimeout in RootSync and RepoSync,
for configuring the threshold for how long to wait for resources in an apply group to reconcile before giving up. An apply group consists of resources without direct or indirect dependencies on each others.
The constraint template library includes a new template: K8sRequiredResources. For reference see Constraint template library.
The template library's K8sProhibitRoleWildcardAccess template now supports regular expression matching of clusterRole names by using the new regexMatch field.
The template library's K8sNoExternalServices template supports a new field: cloudPlatform.
Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 206bbe9).
This release includes several Config Sync performance improvements:
- Config Sync reconciler now watches resources for status updates instead of polling, leading to faster, more responsive, and more efficient detection of object failure and reconciliation. This change also significantly reduces memory allocations for unchanged objects.
- Disable client-side throttling when server-side throttling is enabled (enabled by default on Kubernetes v1.20 and later). This significantly reduced sync latency at scale.
Fixed the issue causing the Config Sync webhook deployment to report readiness prematurely by adding a readiness probe that waits for the TLS certificate to be generated and injected by the sidecar.
Fixed the issue causing git-importer to wipe out non-blocking validation errors before retrying.
Fixed the issue causing reconciler to throw an error when deleting an object that was already deleted.
Fixed two issues when resources are marked unmanaged using the configmanagement.gke.io/managed: disabled annotation:
- If a repo contains unmanaged resources on the initial sync, the sync fails.
- If a resource was deleted in the same commit where another resource is marked unmanaged, the deleted resource is not properly pruned.
May 19, 2022
Fixed metrics to use correct reconciler Pod name for multiple RootSync and RepoSync objects. The metrics are documented at Config Sync metrics
April 21, 2022
Added support for using Fleet Workload Identity to authenticate to Git repositories in Cloud Source Repositories. To learn more, see Grant Config Sync read-only access to Git.
Added a new --timeout flag to the nomos bugreport command. This flag configures the timeout for connecting to the cluster.
ConfigSync ignores the hidden directories .github, .gitlab, and the hidden file .gitlab-ci.yml.
Fixed the issue where nomos bugreport blocks on IO when the number of managed resources exceeds the buffer limit.
March 24, 2022
New Nomos CLI ARM binaries for Linux and Mac (darwin) are now available. Users on these platforms can now install with gcloud components install nomos. The new binary versions will be included in Anthos Config Management downloads.
You can now specify apply and delete ordering using the new config.kubernetes.io/depends-on annotation. To learn more, see Declare resource dependencies between resource objects.
Added dependency enforcement to skip apply of objects whose dependencies are not successfully applied and fully reconciled.
Added support for multiple RootSync objects on the same cluster and multiple RepoSync objects in the same namespace. To learn more, see Configure syncing from multiple repositories.
Added new fields to the ResourceGroup inventory object to distinguish between actuation status and reconciliation status, as well as the intended actuation strategy (apply or delete) before actuation.
Updated the spec.override.resources field on RootSync and RepoSync to let you override the default resource amounts (for example, CPU or memory) requested by the corresponding containers of the reconciler Deployment. To learn more, see Troubleshoot Config Sync.
Increased the config-management-operator container memory request to 100Mi.
Increased the reconciler-manager container memory request to 50Mi and memory limit to 200Mi.
Increased the admission-webhook container memory request to 100Mi and memory limit to 250Mi.
Fixed individual resource object status to correspond to actuation and reconciliation of the latest specification from Git, not just reconciliation status of the last specification that was successfully applied.
February 24, 2022
The constraint template library includes new templates: K8sPSPAutomountServiceAccountTokenPod, RestrictNetworkExclusions, and K8sDisallowAnonymous.
The template library's K8sContainerRatios template supports a new field: cpuRatio.
The template library's K8sRestrictRoleBindings template now supports regular expression matching of role/clusterRole names by using the regexMatch field.
The template library's K8sProhibitRoleWildcardAccess template now allows roles and clusterRoles specified in the constraint to be exempted from the policy.
A set of template library's templates now include the exemptImages parameter, which exempts specific containers from the policy. Those templates are:
- K8sPSPAllowPrivilegeEscalationContainer
- K8sPSPAppArmor
- K8sPSPCapabilities
- K8sContainerLimits
- K8sContainerRatios
- K8sPSPHostNetworkingPorts
- K8sImageDigests
- K8sPSPPrivilegedContainer
- K8sPSPProcMount
- K8sPSPReadOnlyRootFilesystem
- K8sPSPSeccomp
- K8sPSPSELinuxV2
- K8sPSPAllowedUsers
- K8sContainerLimits
Fixed an issue in the hydration-controller container causing the reconciler Pod crash looping when there is a malformed or missing kustomization.yaml in the base directory.
Fixed a memory leak in the Config Sync reconciler container that led to high memory utilization or Pod restarts due to out-of-memory errors.
January 27, 2022
Eliminated duplicate metrics and reduced nonessential tags. For a full list of metrics, see Monitor Config Sync in multi-repo mode.
Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: a478ae6).
This release note was updated on January 31, 2022. The update removed information about two new templates K8sPSPAutomountServiceAccountTokenPod and RestrictNetworkExclusions that are not yet available.
This release note was updated on January 31, 2022. The update removed information about a new field cpuRatio that is not yet available.
This release note was updated on January 28, 2022. The update removed information about a change to the K8sRestrictRoleBindings template that is not yet available.
Fixed the issue in legacy mode (not using RootSync and RepoSync APIs) where unmanaged namespaces (with the configmanagement.gke.io/managed:disabled annotation) were deleted in the cluster when they were deleted from the repository.
Fixed the issue where nomos bugreport could leak Git proxy credentials if present in the git-sync ConfigMap.
December 09, 2021
This note was updated on December 14, 2021:
Config Sync admission webhook is disabled by default. In addition, you can configure to enable and disable Config Sync admission webhook. Config Sync blocks drifts in the cluster when the admission webhook is enabled. To learn more, see Prevent config drift.
- If you install Config Sync using Google Cloud Console or gcloud, you can enable the webhook by adding the following setting to your
apply-spec.yamlconfiguration file:spec.configSync.preventDrift: true(requires Cloud SDK 367.0.0 or later) - If you install Config Sync manually with
kubectl, you can enable the webhook by adding the following setting to yourconfig-management-operator.yamlconfiguration file:spec.preventDrift: true
Policy Controller has deprecated the K8sPSPSELinux (v1) ConstraintTemplate. The K8sPSPSELinuxV2 template has been available since Anthos Config Management 1.5.2. Constraints created against K8sPSPSELinux (v1) are not compatible with the K8sPSPSELinuxV2 template. Customers using constraints based on the K8sPSPSELinux (v1) template will need to recreate those constraints against K8sPSPSELinuxV2.
The Config Sync feature to render Kustomize configurations and Helm charts is generally available (GA). To learn more, see Use a repo with Kustomize configurations and Helm charts.
The Policy Controller feature to support mutation is generally available (GA). To learn more, see Mutate Resources.
Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: c36e3d8).
Fixed the issue causing some resources not to be applied when the status updates of all the resources in a Git repository take longer than 1 minute.
Fixed the issue in RootSync and RepoSync APIs causing proxy to incorrectly fail validation when auth is set to cookiefile or none.
Use nomos migrate to easily enable the RootSync and RepoSync APIs in the cluster. These APIs provide you with additional features and gives you the flexibility to sync to a single repository, or multiple repositories.
Added a new metric pipeline_error_observed to capture if there is any error from different stages: rendering, sync, source, readiness.
nomos status surfaces messages from resource conditions when the managed resources are not ready or healthy.
Increased memory request for git-sync container to 200Mi.
Fixed the issue causing nomos hydrate not to render Kustomize configs if it references files in parent directories.
Fixed the issue causing nomos vet --namespace to fail because it incorrectly defaults --source-format to hierarchy.
This note was updated on December 14, 2021:
Reduced the latency to sync a root repository in RootSync and RepoSync APIs by reducing GET calls to the API server.
October 28, 2021
Config Sync will not block deletion requests if the object has non-nil metadata.deletionTimestamp.
Increased git-importer memory limit to 500Mi.
Fixed the issue causing nomos hydrate not to generate the configurations for clusters selected by the configsync.gke.io/cluster-name-selector annotation.
Fixed the issue causing nomos hydrate to incorrectly require cluster objects to exist in the clusterregistry directory for unstructured repositories.
Fixed the issue causing the namespace to be only synced to one of the clusters when the config for a namespace is defined multiple times with different configmanagement.gke.io/cluster-selector or configsync.gke.io/cluster-name-selector annotations.
September 23, 2021
Config Sync supports rendering Kustomize configurations and Helm charts in multi-repo mode. The Git repository must have a kustomization.yaml file in the root of the sync directory to trigger the rendering process. To learn more, see Use a repo with Kustomize configurations and Helm charts.
The nomos hydrate command supports rendering unstructured source format and it supports rendering Kustomize configurations or Helm charts.
The nomos vet command supports rendering and it supports rendering Kustomize configurations or Helm charts. It provides a --keep-output flag to preserve the rendered output.
Config Sync ignores validating and applying any resource configuration in the Git repo with the annotation config.kubernetes.io/local-config: "true".
When encountering KNV1021: UnknownObjectError, Config Sync applies other resources that aren't affected by this error.
Updated Config Sync CPU requests to fit inside a default GKE cluster and for better resource utilization.
We strongly recommend that all Config Sync users enable multi-repo mode. It provides you with additional features and gives you the flexibility to sync to a single repository, or multiple repositories. If you are using kubectl to install and manage Config Sync, you can enable multi-repo mode by setting spec.enableMultiRepo: true in your ConfigManagement object. For more details, see Syncing from multiple repositories.
The Anthos Config Management operator is now installed into the config-management-system namespace rather than the kube-system namespace. If you are running custom monitoring or installation processes you need to update those processes. For specific instructions, see Manually installing Config Sync and Policy Controller with kubectl.
In nomos versions earlier than 1.9.0, the nomos status command reports an incorrect status for clusters using an Anthos Config Management version of 1.9.0 or later. Before upgrading to Anthos Config Management 1.9.0 or later, download the latest nomos CLI tool.
Fixed the issue causing the reconciler image version not getting updated, when upgrading from Anthos Config Management version 1.6.2. This was caused by an immutable label added in Anthos Config Management 1.6.2 and removed in 1.7.0.
August 26, 2021
kube-rbac-proxy has been removed since Hierarchy Controller does not expose any sensitive metrics, and kube-rbac-proxy is no longer actively maintained.
Fixed the issue causing a root or namespace reconciler to fail to be updated after switching from the auth type from none|gcenode|gcpserviceaccount to other types.
Fixed the issue causing Config Sync not to report sync errors when it fails to access Git repos.
Config Sync supports storing HTTPS/HTTP proxy credentials inside the git-creds Secret, using https_proxy or http_proxy as a key, to avoid exposing these credentials as plaintext.
Fixed the issue causing Config Sync not to pick up the latest schema of the CustomResourceDefinitions synced from Git repos.
Config Sync provides a way for users to override some system values:
- Use the
spec.override.resourcesfield of a RootSync or RepoSync object to override the resource limits for thereconcilercontainer and thegit-synccontainer. - Use the
spec.override.gitSyncDepthfield of a RootSync or RepoSync object to override the number of git commits to fetch from the git repository. - Set the
spec.git.noSSLVerifyfield of a RootSync or RepoSync object totrueto disable Git SSL certificate verification.
Anthos Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 07e2fd0).
July 22, 2021
This note was updated on August 5, 2021: the issue in the ResourceGroup Controller was fixed.
An issue introduced in 1.8.0 nomos hydrate that breaks support for --clusters has been fixed.
An issue that caused Config Sync monitoring Pods fail to start in a cluster with PodSecurityPolicy enabled has been fixed.
Cluster selectors and namespace selectors annotations are removed from the result of nomos hydrate so that it can pass nomos vet and can be synced directly to the cluster by Config Sync.
An issue in ResourceGroup Controller that failed to update statuses of the managed resources has been fixed.
July 01, 2021
This release note was updated on September 29, 2021:
- The issue with Config Sync causing excessive updates of resources has been fixed.
This release note was updated on August 5, 2021:
- The issue in
nomos hydratehas been fixed. - A bug in
nomos hydratehas been identified. nomos statushas been updated to show resource level status when MultiRepo is enabled.
This release note contains information about 1.8.0 features that are now more widely available.
Config Sync now supports accessing Cloud Source Repositories through a Google service account when Workload Identity is enabled in your cluster. To learn more, see Granting Config Sync read-only access to Git.
Config Management is now available on GKE. Config Management enables you to use Policy Controller. GKE users can also now install Config Sync using the Cloud Console or by using the gcloud command-line tool. To learn more, see Installing Config Sync.
The following commands have been promoted to beta:
gcloud container hub config-management applygcloud container hub config-management disablegcloud container hub config-management enablegcloud container hub config-management statusgcloud container hub config-management unmanagegcloud container hub config-management upgradegcloud container hub config-management version
The config file format for the gcloud apply command has changed. For more information on the new file format, see gcloud apply spec fields.
You can now configure your cluster with the same settings used by another cluster by using gcloud fetch-for-apply. To learn more, see Configuring Config Sync.
Config Sync cluster selectors support CustomResourceDefinitions.
The issue where nomos hydrate command attempts to connect to the API Server even if --no-api-server-check is passed has been fixed.
A bug in nomos hydrate breaks support for --clusters.
nomos status shows resource level status when MultiRepo is enabled.
June 24, 2021
Config Connector can no longer be installed via Anthos Config Management. Upgrading to Anthos Config Management v1.8.0 will not affect an existing Config Connector installation, but that installation can no longer be managed with Anthos Config Management. To install or upgrade Config Connector alongside Anthos Config Management v1.8.0 or later, see Advanced installation options > Upgrading from non-operator installations in the Config Connector documentation. The version of Config Connector supported in earlier versions of Anthos Config Management will stop working on Kubernetes versions greater than or equal to 1.19.
The Config Sync admission webhook serving port is switched from 8676 to 10250. If you use Config Sync in multi-repo mode in private GKE clusters, you no longer need to add a firewall rule to open port 8676.
The Hierarchy Controller admission webhook serving port has switched from 9443 to 10250. If you use Hierarchy Controller in private GKE clusters you no longer need to add a firewall rule to open port 9443.
The Anthos Policy Controller admission webhook serving port is switched from 8443 to 10250. If you use Policy Controller in private GKE clusters you no longer need to add a firewall rule to open port 8443.
All Anthos Config Management components have been updated to remove use of v1beta1 APIs scheduled to be removed in Kubernetes 1.22. See the Kubernetes Deprecated API Migration Guide for more details.
Anthos Policy Controller now supports the ability for users to mutate resources as a preview feature. For more information see Mutating resources.
Anthos Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: f6c2fe8).
Editing rights to Hierarchical Resource Quotas are now aggregated into the cluster-wide 'edit' and 'admin' Cluster Roles.
May 26, 2021
The output of the nomos hydrate command does not pass nomos vet and cannot be synced using Config Sync without modifying the output. To work around this, we recommend removing the following annotations: configmanagement.gke.io/cluster-name , configmanagement.gke.io/source-path and removing label configsync.gke.io/declared-version
from the output so that the output can be successfully synced.
The nomos hydrate command attempts to connect to the API Server even if --no-api-server-check is passed. This behavior can be safely ignored in CI as if the CLI is unable to connect to the API Server it will not produce errors resulting from being unable to connect.
Hierarchy Controller has been updated to use HNC v0.8.0.
Increased reconciler memory limit to 300Mi.
May 20, 2021
If Syncing from multiple repositories is enabled on a private GKE cluster, it's required to add a firewall rule to allow port 8676.
May 13, 2021
Anthos Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 9b5e4cf).
A bug in Anthos Config Management 1.7.0 which broke nomos hydrate --no-api-server-check has been fixed.
The Config Sync admission webhook in Anthos Config Management 1.7.0 would block requests when a managed resource in the cluster copied annotations to another resource.
Config Sync container images are now correctly updated when Anthos Config Management is upgraded.
A bug in Anthos Config Management 1.7.0 which caused nomos status to return errors when both unstructured repos and Hierarchy Controller were being used has been fixed.
April 13, 2021
Anthos Config Management v1.7.0 included several Kubernetes library updates, one of these updates made checks for Resource types more strict. As a consequence, Config Sync users upgrading from an older version of Anthos Config Management may see errors in the form KNV9998: failed to encode declared fields: internal error: ....resources.limits.cpu: expected string, got &value.valueUnstructured{Value:2}. As a workaround, all resource declarations should be specified as strings.
April 05, 2021
This release note was updated on September 29, 2021:
- An issue with Config Sync causing excessive updates of resources has been identified.
Anthos Config Management images are no longer included in Anthos on VMWare clusters. To learn more, see Changes to Anthos Config Management updates.
The ability to sync from multiple Git repositories is now a generally-available feature. To learn more, see Syncing from multiple repositories.
A memory leak in the Anthos Config Management Operator Pod that led to high memory utilization or Pod restarts due to out-of-memory errors has been corrected.
Preview versions of multi-repo occasionally used excessive CPU usage and sent unnecessary queries to the apiserver master node, resulting in an unhealthy cluster. This issue has been corrected.
Config Sync configured with sourceFormat: unstructured will have errors during syncing if the Git repository includes a "Repo" resource.
Config Sync configured with sourceFormat: unstructured will have errors during syncing if the Git repository specifies a ClusterSelector with an invalid metadata.name field.
Config Sync multi-repo mode can't update the reconciler deployment image version when upgrading from 1.6.2 or later because of a new label removed from the immutable label selector. The workaround is to manually delete the reconciler by running kubectl delete deployment -n config-management-system -l app=reconciler. For more information, see Error: reconciler deployments failed to be upgraded.
This release note was updated on August 27, 2021. The update adds information about how to resolve an upgrade issue.
Customers using Anthos Policy Controller who have upgraded since Anthos Config Management 1.5.1 need to update the timeoutSeconds in their ValidatingWebhookConfigurations from "5" to "3" to avoid issues with Kubernetes leader elections.
February 25, 2021
Hierarchy Controller now includes a preview of Hierarchical Resource Quotas (HRQs). HRQs are drop-in replacements for Kubernetes Resource Quotas, but apply to resources in both a namespace as well as all of its descendants. To learn more, see Using hierarchical resource quotas.
The Anthos Config Management Operator Deployment now specifies resources.limits for config-management-operator:manager.
This release note was updated on March 5, 2021. The update removed information about a feature that is not yet available.
Config Sync multi-repo mode can't sync Git repositories using ssh as the authentication method. If this issue affects you, please contact support so we can suggest ways to handle your required use cases while we correct this issue.
This release note was updated on April 24, 2021. The update adds information about how to resolve an issue.
Config Sync multi-repo mode can't update the reconciler deployment image version in the upgrade process because of a new label added to the immutable label selector. The workaround is to manually delete the reconciler by running kubectl delete deployment -n config-management-system -l app=reconciler. For more information, see Error: reconciler deployments failed to be upgraded
.
This release note was updated on August 27, 2021. The update adds information about how to resolve an upgrade issue.
January 28, 2021
Anthos Config Management Operator and Config Sync Pods get high memory utilization or OOMKilled due to memory leak.
Hierarchy Controller is upgraded to include HNC v0.7.0. This release introduces Exceptions. Exceptions let you use Kubernetes label selectors to precisely control where certain objects are propagated.
This release also removes support for the v1alpha1 API. If you were using Hierarchy Controller 1.5.1 or earlier, you must either update to Hierarchy Controller 1.5.2 or 1.6.0, and follow the HNC v0.6.0 directions to upgrade to v1alpha2.
The nomos status output has been fixed for multi-repo clusters to show git.syncBranch when git.syncRev is not specified (git.syncRev defaults to HEAD) to provide a consistent experience with mono-repo clusters.
The nomos status output has been fixed for multi-repo clusters to distinctly show status of multiple namespace repos synced to the clusters.
January 08, 2021
Config Sync unintentionally started using the absolute path in the file system with spec.git.policyDir. This has no effect on Config Sync running on the cluster, but breaks validation when running nomos vet manually against hierarchical repositories. The issue will be corrected in 1.6.1.
December 10, 2020
Anthos Policy Controller now includes additional policies covering many of the CIS Kubernetes Benchmark 1.5.1 controls. To learn more, see the Constraint template library.
Anthos Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 1de87b6).
Support for Git submodules has been fixed in this version.
November 16, 2020
A leading forward slash "/" in spec.git.policyDir will not match anything in Config Sync versions 1.5.2 and 1.6.0. For example:
policyDir: "/dirname"
will not match a top-level directory dirname. Because policyDir is always applied at the top of the directory structure, the workaround is to remove the leading forward slash.
This issue has been corrected in Config Sync 1.6.1; the forward slash will be ignored as it was in 1.5.1 and previous versions.
There is an issue where enabling referential constraints does not take effect for the audit container. This affects Anthos Config Management versions 1.5.0, 1.5.1, and 1.5.2.
The default timeout for Policy Controller's ValidatingWebhookConfiguration has been reduced to avoid interference with Kubernetes leader election processes.
Hierarchy Controller is upgraded to include HNC v0.6.0. This release introduces support for v1alpha2, and will automatically update all your existing HNC objects. We recommend backing up these objects before upgrading in case there are any problems with the upgrade process. For more information, see the release notes for HNC v0.6.0.
October 29, 2020
Anthos Config Management now includes the ability to sync from multiple Git repositories. This is a preview feature. To learn more, see Syncing from multiple repositories.
The installed product version was being misreported as "anthos-config-management" in ACM 1.5.0. The correct product version is now being reported.
The nomos status output has been modified significantly to provide a consistent experience for both mono-repo and multi-repo clusters.
When the enableLegacyFields is set to true, the ACM operator will create a RootSync resource automatically, but any subsequent changes to the RootSync resource will not be noticed by the operator. This will be fixed in a subsequent release. As a workaround, if the RootSync resource resource is modified, add or modify an unused annotation on the ConfigManagement resource to cause the operator to reconcile changes in the RootSync resource.
(Fixed on October 30, 2020) The version of Anthos Configuration Management included in the Anthos On-Prem release 1.5.1-gke.8 had initially referenced a version of the nomos image that had not be moved into the gcr.io/gke-on-prem-release repository, thus preventing a successful installation and/or upgrade of Anthos Configuration Management. This image has since been pushed to the repository to correct the issue for customers not using private registries. Customers using private registries will need to upgrade to 1.5.2 when it is available, or manually copy the nomos:v1.5.1-rc.7 image into their private repository.
September 24, 2020
Anthos Config Management now includes Config Connector v1.19.1.
Anthos Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 15d56e3).
Binary Authorization can now be enabled through the Anthos Config Management Operator. See Setting up with Anthos Config Management for details.
The syncer and importer Containers now both run in the git-importer Pod in the importer Container. Any resource patches that were previously targeted to the syncer container will need to be updated to target the importer container.
Anthos Config Management installs a resource-group-controller Deployment which fails on non-GKE clusters. This Deployment is unnecessary and does not cause any other issues.
The nomos CLI tool is now available via gcloud. Please see the downloads page for more information.
This release includes several logging and performance improvements.
August 27, 2020
Anthos Config Management now includes Config Connector v1.15.1.
Anthos Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 1de87b6).
An issue with git submodule support is preventing syncing of configuration stored in submodule repositories. If this affects you, please contact support so we can suggest ways to handle your required use cases while we correct this.
This release includes several logging and performance improvements.
July 30, 2020
Updated the git-sync image to fix security vulnerability CVE-2019-5482.
July 23, 2020
Config Connector has been updated in Anthos Config Management to version 1.13.1.
Anthos Config Management now includes Hierarchy Controller as a beta feature. For more information on this component, see the Hierarchy Controller overview.
Policy Controller users may now enable --log-denies to log all denies and dryrun failures. This is useful when trying to see what is being denied or fails dry-run and for keeping a log to debug cluster problems without looking through the status of all constraints. This is configured by setting spec.policyController.logDeniesEnabled: true in the configuration file for the Operator. There is an example in the section on Installing Policy Controller.
This release includes several logging and performance improvements.
This release includes several fixes and improvements for the nomos command line utility.
The use of unsecured HTTP for GitHub repo connections or in an http_proxy is now discouraged, and support for unsecured HTTP will be removed in a future release. HTTPS will continue to be supported for GitHub repo and local proxy connections.
This release improves the handling of GitHub repositories with very large histories.
Prior to this release, Config Sync and kubectl controllers and processes used the same annotation (kubectl.kubernetes.io/last-applied-configuration) to calculate three-way merge patches. The shared annotation sometimes resulted in resource fights, causing unnecessary removal of each other's fields. Config Sync now uses its own annotation, which prevents resource clashes.
In most cases, this change will be transparent to you. However, there are two cases where some previously unspecified behavior will change.
The first case is when you have run kubectl apply on an unmanaged resource in a cluster, and you later add that same resource to the GitHub repo. Previously, Config Sync would have pruned any fields that were previously applied but not declared in the GitHub repo. Now, Config Sync writes the declared fields to the resource and leaves undeclared fields in place. If you want to remove those fields, do one of the following:
- Get a local copy of the resource from GitHub and
kubectl applyit. - Use
kubectl edit --save-configto remove the fields directly.
The second case is when you stop managing a resource on the cluster or even stop all of Config Sync on a cluster. In this case, if you want to prune fields from a previously managed resource, you will see different behavior. Previously, you could get a local copy of the resource from GitHub, remove the unwanted fields, and kubectl apply it. Now, kubectl apply no longer prunes the missing fields. If you want to remove those fields, do one of the following:
- Call
kubectl apply set-last-appliedwith the unmodified resource from GitHub, then remove unwanted fields andkubectl applyit again without theset-last-appliedflag. - Use
kubectl edit --save-configto remove the fields directly.
In error messages, links to error docs are now more concise.
June 25, 2020
Anthos Config Management is now Generally Available on AKS (Kubernetes v1.16 or higher) and EKS (Kubernetes v1.16 or higher).
Config Connector is not currently supported on EKS or AKS, as it is unable to run on these providers.
The following Policy Controller constraint templates have been added to the Default Template Library:
- allowedserviceportname
- destinationruletlsenabled
- disallowedauthzprefix
- policystrictonly
- sourcenotallauthz
The following constraint templates have been updated:
- k8sblockprocessnamespacesharing
- k8sdisallowedrolebindingsubjects
- k8semptydirhassizelimit
- k8slocalstoragerequiresafetoevict
- k8smemoryrequestequalslimit
- k8snoexternalservices
- k8spspallowedusers
- k8spspallowprivilegeescalationcontainer
- k8spspapparmor
- k8spspcapabilities
- k8spspflexvolumes
- k8spspforbiddensysctls
- k8spspfsgroup
- k8spsphostfilesystem
- k8spsphostnamespace
- k8spsphostnetworkingports
- k8spspprivilegedcontainer
- k8spspprocmount
- k8spspreadonlyrootfilesystem
- k8spspseccomp
- k8spspselinux
- k8spspvolumetypes
See the Default Template Library documentation for more information.
Anthos Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 25ca799).
This new build of OPA Gatekeeper includes a number of bug fixes and performance improvements, and adds three new monitoring metrics:
- gatekeeper_sync
- gatekeeper_sync_duration_seconds
- gatekeeper_sync_last_run_time
The nomos CLI tool now supports the KUBECONFIG environment variable in a way that matches the kubectl behavior with multiple delimited configuration files.
Anthos Config Management no longer gets into a continuous PATCH loop when encountering unmanaged resources with config-management annotations and a missing last-applied-configuration annotation.
Anthos Config Management is not issuing errors when it encounters certain types of malformed configurations in a resource definition. This may result in the Kubernetes API Server ignoring the malformed fields and applying the default value for the field instead.
Policy Controller may fail to start successfully when synced resources are marked for deletion.
This issue will be addressed in the upstream OPA Gatekeeper project in a future release. For more information see the relevant issue in the Gatekeeper project.
This release includes several logging and performance improvements.
June 15, 2020
A regression in Anthos Config Management 1.3.2 results in unnecessary patches to the API server for the gatekeeper-system namespace and spurious logging for error KNV2005. This "fight" results when the gatekeeper-system namespace is managed in the Git repo, and two Anthos Config Management components (the operator and syncer) are both trying to reconcile the state of the namespace with the API server. The only workaround at this time is to unmanage the gatekeeper-system namespace. The issue will be fixed in Anthos Config Management 1.4.1.
May 21, 2020
This release includes several performance and memory improvements.
In order to help prevent accidental deletion, Anthos Config Management will no longer allow a user to remove all namespaces or cluster-scoped resources in a single commit. If you wish to delete the full set of resources under management, it now requires two steps: remove all but one in a first commit, allow ACM to sync those changes, then remove the final resource in a second commit.
Error documentation has been updated to add more information on error codes. Errors that are no longer encountered in the product have been removed. Most error references have been embellished with examples and steps for remediation.
Anthos Config Management now supports a GKE-only authentication mechanism based on the service account of the cluster's node pool. Documentation on its use is here.
Anthos Config Management now includes Config Connector v1.8.0.
Anthos Config Management will now attempt to detect when resources that it manages are also managed by other controllers. Documentation on this behavior is available in error knv2005 which ACM will log in that case.
Policy Controller has been upgraded to include a newer version of Open Policy Agent Gatekeeper.
This version includes updates to improve the management of policy resources. As a consequence, finalizers are no longer used to manage Constraints and Constraint Templates.
The following metrics have been made obsolete due to these changes and have been removed:
gatekeeper_watch_manager_is_running
gatekeeper_watch_manager_last_restart_check_time
gatekeeper_watch_manager_last_restart_time
gatekeeper_watch_manager_restart_attempts
The following metrics were removed and will be re-implemented in a later version:
gatekeeper_watch_manager_intended_watch_gvk
gatekeeper_watch_manager_watched_gvk
April 23, 2020
Anthos Config Management images are now included in the Google-provided system images for Binary Authorization.
Policy Agent now allows configuration of namespaces that will bypass the admission controller. For more information, see Excluding Namespaces from Policy Controller
You can now exempt Namespaces from Policy Controller enforcement
Anthos Config Management v1.3.1 now supports Kubernetes v1.16 and higher. Earlier versions of Anthos Config Management relied on APIs that have been deprecated in Kubernetes v1.16.
The Anthos Config Management Syncer pod now reports when it detects that it is fighting with another process over a resource.
Anthos Config Management no longer allows managing resources in unmanaged Namespaces.
If you define a CRD with an integer field that has min/max values, Anthos Config Management will be unable to update the CRD.
Anthos Config Management no longer overwrites undeclared labels and annotations on Namespaces.
March 24, 2020
Anthos Policy Controller is now Generally Available
Anthos Config Management now includes the generally-available version of Config Connector.
Anthos Config Management now supports the use of a Personal Access Tokens for authentication against supported Git providers. More information can be found in Installing Anthos Config Management.
Anthos Config Management now supports the use of an HTTP or HTTPS proxy to connect with your Git host. More information can be found at Installing Anthos Config Management.
February 21, 2020
GKE On-prem 1.2.2 includes images for ACM 1.2.1. Upgrading from ACM 1.2.1 to ACM 1.3 is a valid, tested, safe upgrade path.
Minor updates and bug fixes.
February 10, 2020
It is not currently possible to downgrade to v1.0.x after upgrading to a more recent version of Anthos Config Management.
Currently, Config Connector can only be installed on a single cluster. If multiple Config Connector instances attempt to create or mutate the same Google Cloud resource, conflicts or errors may occur, and you may exhaust quota for a given resource.
Anthos Config Management now can optionally support an unstructured repository, though some features that relied on hierarchical namespaces are disabled in this mode. More information can be found here.
Anthos Config Management v1.2.1 is generally available for use in production.
Git repos with submodules are now also supported by Anthos Configuration Management out of the box without additional configuration. This allows delegation of config authoring in a Git-idiomatic way. For more information, please see Git's documentation on submodules.
A new CLI subcommand is available. nomos bugreport bundles up Anthos Config Management log information into a Zip file, which can be attached to a Google support ticket.
Previously, adding an APIService to the repo will leave Anthos Config Management in a bad state, with the error message "KNV2002: failed to get server resources: unable to retrieve the complete list of server APIs." This issue has been mitigated; Anthos Config Management will now sync APIService objects correctly.
December 20, 2019
Anthos Config Management v1.2.0 is generally available for use in production.
This release has minor bug fixes and performance improvements.
September 19, 2019
Anthos Config Management v1.1.0 is generally available for use in production.
Policy Controller (Beta) is a Kubernetes dynamic admission controller that checks, audits, and enforces your clusters' compliance with policies related to security, regulations, or business rules. It is built using Gatekeeper, an open source project.
Adding an APIService to the repo will leave Anthos Config Management in a bad state, with the error message KNV2002: failed to get server resources: unable to retrieve the complete list of server APIs." This issue will affect both new and existing clusters syncing from this repo. To correct the issue:
- find the name of the
git-importerandsyncerpods usingkubectl get pods -n config-management-system - copy those names and restart the pods with
kubectl delete -n config-management-system pods git-importer-xxxx-xxxx syncer-xxxx-xxxx - These steps are required once for each cluster.
Once the pods for a cluster are restarted, syncing will be back to normal.
You can now enable integration with Config Connector (beta), a Kubernetes addon that allows you to manage your Google Cloud Platform resources through Kubernetes configuration. You can sync configs for Google Cloud Platform resources with your Anthos Config Management repo and apply them automatically. For more information, see Installing Config Connector.
The apiVersion for the ConfigManagement CustomResource has changed. No action is required; the CustomResource is upgraded automatically when you upgrade to v1.1.0. You can read more about configuring Anthos Config Management.
Managed CRDs (CustomResourceDefinitions) are now Namespace-scoped by default, instead of cluster-scoped. This matches the semantics of the kubectl command.
If a CRD explicitly specifies a scope, Anthos Config Management honors that scope.
The nomos hydrate command is a replacement for the nomos view command, and reports your Anthos Config Management configuration in a human-readable way.
To use nomos hydrate, upgrade the nomos command to v1.1.
If you need to continue using nomos view, do not upgrade the nomos command from v1.0. It will remain forward-compatible for the foreseeable future.
You can read about a known issue with nomos view.
Anthos Config Management can now be installed on clusters using PodSecurityPolicies.
The nomos view command is deprecated and is not included in v1.1 or higher of the nomos command.
If you need to continue using nomos view, do not upgrade the nomos command from v1.0. It will remain forward-compatible for the foreseeable future.
It is not currently possible to downgrade to v1.0.x after upgrading to v1.1.0.
Currently, Config Connector can only be installed on a single cluster. If multiple Config Connector instances attempt to create or mutate the same Google Cloud Platform resource, conflicts or errors may occur, and you may exhaust quota for a given resource.
nomos view can fail to parse CRDs (Custom Resource Definitions) that exist in the local clone of the repo but have not yet been synced to a cluster.
To work around this issue, use nomos hydrate instead of nomos view.
June 14, 2019
Anthos Config Management v1.0.0 is generally available for use in production.
To upgrade to this version, follow the instructions for upgrading.
You must update all nomos binaries when you upgrade to Anthos Config Management v1.0.0.
Versions older than v1.0.0 are no longer available. If you participated in the early-access program for Anthos Config Management, you must upgrade to v1.0.0.
You can now sync CustomResourceDefinitions (CRDs). Anthos Config Management can now sync any type of Kubernetes object. For more information, see Configuring CustomResourceDefinitions.
HierarchicalResourceQuotas are no longer supported.
We document how to stop Anthos Config Management from syncing configs as quickly as possible. This allows you to mitigate the potential for propagating unintended configs to clusters.
The nomos status subcommand provides a top-level view of the state of Anthos Config Management on all enrolled clusters, including errors and sync status. It reports on all clusters that kubectl can access.
The product name is now Anthos Config Management.
The nomos version command now provides version details for the Config Management Operator on each configured cluster, as well as the version of the nomos command itself.
New metrics allow you to monitor counts, latencies, and timestamps of specific operations.
The following changes have been made to the canonical example repository:
The canonical example repo has moved. If you use this repo or a fork, update your Git repository's remotes or create a separate clone of the new repo to ensure that you receive updates.
The filesystem standard and the value of the Repo object's spec.version for this version of Anthos Config Management are both
1.0.0.A new branch named
1.0.0contains configs compatible with Anthos Config Management v1.0.0.
An example NetworkPolicy illustrates some methods for enforcing good security practices across your clusters.
We improved the instructions for setting up SSH keys for authenticating to a Git repository.
March 29, 2019
Anthos Config Management v0.13.1 (beta) is a maintenance release, and is compatible with Anthos Config Management v0.13.0.
To upgrade from v0.13.0 on an existing cluster, deploy the new Config Management Operator. You may need to remove an object that is no longer used, to prevent spurious log messages. See the release note about the change below.
You can now manage the default Namespace as well as Namespaces with names beginning with kube-.
Previously, if a config change removed a controller object (for example, a Deployment that has a ReplicaSet), removing the controller object did not remove objects it controlled. All of a controller object's child objects are now correctly removed when the controller object itself is removed.
Previously, if your repo contained only configs for Namespaces and no other configs, the Namespace configs would fail to sync. Repos now sync correctly even if it only contains configs for Namespaces.
The git-policy-importer application has been renamed to git-importer.
The nomos-cluster-policy ClusterConfig has been renamed to config-management-cluster-config. After upgrading, both ClusterConfig objects both exist on the cluster. This does not impact the functionality of the cluster, but you may see spurious log messages if the older ClusterConfig is still present. You can remove the old ClusterConfig to avoid these log messages:
kubectl delete clusterconfig nomos-cluster-policy
Syncing of CustomResourceDefinitions is not currently supported. If CustomResourceDefinition has been applied to the cluster, you can sync associated CustomResources.
Update: This issue no longer exists in Anthos Config Management v1.0.0 and higher.
March 20, 2019
The nomos-system Namespace has been renamed to config-management-system.
The nomos.dev/ API group has been renamed to configmanagement.gke.io/.
The Nomos object has been renamed to the ConfigManagement object and is now cluster-scoped.
The nomos-resource-quota object, which combines all of a Namespace's effective ResourceQuotas into a single one that is more efficient for Kubernetes to check and enforce, has been renamed to config-management-resource-quota.
Prometheus now uses the gkeconfig Namespace.
Annotations, rather than labels, are now used to determine that Anthos Config Management is managing a Kubernetes object.
Syncing of CustomResourceDefinitions is not currently supported. If a CustomResourceDefinition has been applied to the cluster, you can sync associated CustomResources.
Update: This issue no longer exists in Anthos Config Management v1.0.0 and higher.
Anthos Config Management 13.0.0 is the second beta release of Anthos Config Management. It represents a major change from v0.11.6, is not backward-compatible with any prior release, and cannot be installed on a cluster with a previous installation of Anthos Config Management. Backward-incompatible releases will always use a new minor version number.
You can now share a ResourceQuota among multiple Namespaces with a common abstract namespace directory. See Aggregate ResourceQuotas.
Syncs are no longer required, and are now silently ignored by the Config Management Operator. You can now create a config for any object in your cluster except a CustomResourceDefinition or the Config Management Operator itself.
March 04, 2019
This is the beta release of Anthos Config Management. It represents a major change from v0.10.4, and cannot be installed on a cluster with a previous installation of Anthos Config Management. An existing installation of the alpha from v0.10.4 or earlier will conflict with a new installation of v0.11.6 due to changes in the repository structure.
Added support for syncing all Kubernetes resources generically. For current limitations, see the list of known issues for this release.
Added support for NamespaceSelectors.
Moved repository format to Filesystem Standard v0.1.0.
Moved installation to use the Config Management Operator.
Syncing of CustomResourceDefinitions is not currently supported. If a CustomResourceDefinition has been applied to the cluster, you can sync associated CustomResources.
Update: This issue no longer exists in Anthos Config Management v1.0.0 and higher.
In some cases, local changes to managed resources made by kubectl apply can result in the removal of the nomos.dev/managed: enabled label, causing the resource to become unmanaged. As a workaround, use kubectl edit instead, or include the label in the YAML manifest you are applying.
Update: This issue no longer exists in Anthos Config Management v1.0.0 and higher. If changes are manually applied to managed Kubernetes objects, Anthos Config Management effectively reverts those changes as soon as it notices a difference between the config in the repo and the object in the cluster.
For more information, see Managing and unmanaging objects.