Use Config Sync with Kustomize and Helm

In this tutorial, you add Kustomize configurations that reference Helm charts to your repository and then use Config Sync to sync your cluster to your repository.

When you use Config Sync, the Kustomize configurations and Helm charts you place in your Git repository are automatically rendered. Automated rendering provides you with the following benefits:

  • You no longer need an external hydration pipeline. Without automated rendering, you have to manually render the configurations using Kustomize and Helm on your workstation, or set up a step to trigger the hydration process in your CI systems. With automated rendering, Config Sync handles the execution.

  • Your maintenance costs are reduced. Without automated rendering, you have to maintain one Git repository with the original Kustomize configurations and Helm charts and another Git repository with the output generated by the external hydration. You then have to configure Config Sync to sync from the Git repository with the rendered output. With automated rendering, you only need to maintain one repository with the original configs.

  • Your development workflow is simplified. Without automated rendering, changes made to your original configs need to be reviewed twice before being merged; once in the original repository and again in the rendered repository. With automated rendering, the rendered configs are generated by Config Sync, and you only need to review the changes to the original configs.

Configure your repository

The following tasks show you how to prepare a Git repository with configs that combine Kustomize configurations with Helm charts:

  1. Create, or make sure you have access to, a Git repository. Since your repository uses Kustomize and Helm, this should be an unstructured repository.

  2. In the root of your Git repository, create a file named kustomization.yaml and paste the following code into it:

    # ./kustomization.yaml
    resources:
    - base
    
    patches:
    - path: ignore-deployment-mutation-patch.yaml
      target:
        kind: Deployment
    

    This file is a Kustomize overlay that points to the Kustomize base. This overlay includes a patch for the Helm chart base that adds the client.lifecycle.config.k8s.io/mutation: ignore annotation to all Deployment objects. The annotation causes Config Sync to ignore any conflicting changes to this object in the cluster after you have created it.

  3. In your Git repository, create a directory named base:

    mkdir base
    
  4. In the base directory, create another file named kustomization.yaml and paste the following code into it:

    # ./base/kustomization.yaml
    helmCharts:
    - name: cert-manager
      repo: https://charts.jetstack.io
      version: v1.5.3
      releaseName: my-cert-manager
      namespace: cert-manager
    

    This file is the Kustomize base, which renders the remote Helm chart.

  5. Navigate back to the root of your Git repository, create a file named ignore-deployment-mutation-patch.yaml and paste the following code into it:

    # ./ignore-deployment-mutation-patch.yaml
    apiVersion: apps/v1
    kind: Deployment
    metadata:
     name: any
     annotations:
       client.lifecycle.config.k8s.io/mutation: ignore
    

    This file is a patch that is applied to the base Helm chart. It adds the client.lifecycle.config.k8s.io/mutation: ignore annotation to all Deployments in the base directory.

  6. Commit the changes to your repository:

    git add .
    git commit -m 'Set up manifests.'
    git push
    

The samples repository has an example of what such a repository would look like.

Preview and validate rendered configs

Before Config Sync renders the configs and syncs them to the cluster, ensure that the configs are accurate by running nomos hydrate to preview the rendered configuration and running nomos vet to validate that the format is correct.

  1. Run the following nomos hydrate with the following flags:

    nomos hydrate \
        --source-format=unstructured \
        --output=OUTPUT_DIRECTORY
    

    In this command:

    • --source-format=unstructured lets nomos hydrate work on an unstructured repository. Since you are using Kustomize configurations and Helm charts, you need to use an unstructured repository and add this flag.
    • --output=OUTPUT_DIRECTORY lets you define a path to the rendered configs. Replace OUTPUT_DIRECTORY with the location that you want the output to be saved in.
  2. Check the syntax and validity of your configs by running nomos vet with the following flags:

    nomos vet \
        --source-format=unstructured \
        --keep-output=true \
        --output=OUTPUT_DIRECTORY
    

    In this command:

    • --source-format=unstructured lets nomos vet work on an unstructured repository.
    • --keep-output=true saves the rendered configs.
    • --output=OUTPUT_DIRECTORY is the path to the rendered configs.

Configure syncing from the Git repository

Now that you have created a repository with the configs that you want to use, you can configure syncing from your cluster to your repository.

  1. To configure your RootSync object, create a root-sync.yaml file:

    # root-sync.yaml
    apiVersion: configsync.gke.io/v1beta1
    kind: RootSync
    metadata:
      name: root-sync
      namespace: config-management-system
    spec:
      sourceFormat: unstructured
      git:
        repo: YOUR_GIT_REPOSITORY
        branch: main
        auth: none
      override:
        enableShellInRendering: true
    

    Replace YOUR_GIT_REPOSITORY with the URL of your Git repository.

  2. Apply the root-sync.yaml file to your cluster:

    kubectl apply -f root-sync.yaml
    

Verify the installation

After you have installed and configured Config Sync, you can verify that the installation completed successfully.

  1. Verify that there no other errors by using nomos status:

    nomos status
    

    Example output:

    *CLUSTER_NAME
    --------------------
    <root>   https:/github.com/GoogleCloudPlatform/anthos-config-management-samples.git/helm-component/manifests@init
    SYNCED   fd17dd5a
    
  2. Verify if the Helm component is successfully installed:

    kubectl get all -n cert-manager
    

    Example output:

    NAME                                              READY   STATUS    RESTARTS   AGE
    pod/my-cert-manager-54f5ccf74-wfzs4               1/1     Running   0          10m
    pod/my-cert-manager-cainjector-574bc8678c-rh7mq   1/1     Running   0          10m
    pod/my-cert-manager-webhook-7454f4c77d-rkct8      1/1     Running   0          10m
    
    NAME                              TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
    service/my-cert-manager           ClusterIP   10.76.9.35     <none>        9402/TCP   10m
    service/my-cert-manager-webhook   ClusterIP   10.76.11.205   <none>        443/TCP    10m
    
    NAME                                         READY   UP-TO-DATE   AVAILABLE   AGE
    deployment.apps/my-cert-manager              1/1     1            1           10m
    deployment.apps/my-cert-manager-cainjector   1/1     1            1           10m
    deployment.apps/my-cert-manager-webhook      1/1     1            1           10m
    
    NAME                                                    DESIRED   CURRENT   READY   AGE
    replicaset.apps/my-cert-manager-54f5ccf74               1         1         1       10m
    replicaset.apps/my-cert-manager-cainjector-574bc8678c   1         1         1       10m
    replicaset.apps/my-cert-manager-webhook-7454f4c77d      1         1         1       10m