Understanding Feature State Conditions
Note: Canonical Services are supported automatically in Cloud Service Mesh version 1.6.8 and higher.
This page describes how to interpret and act upon Conditions reported on your Cloud Service Mesh cluster or fleet.
To check for conditions, run this command:
gcloud container fleet mesh describe --project FLEET_PROJECT
The output may include conditions in membershipStates for a cluster, like:
...
membershipStates:
projects/test-project/locations/us-central1/memberships/my-membership:
servicemesh:
conditions:
- code: <CONDITION_CODE>
details: ...
documentationLink: ....
severity: ...
Values of code are described in more detail on this page.
NODEPOOL_WORKLOAD_IDENTITY_FEDERATION_REQUIRED
You may see the NODEPOOL_WORKLOAD_IDENTITY_FEDERATION_REQUIRED error code in
the Conditions field of your membership:
membershipStates:
projects/test-project/locations/us-central1/memberships/my-membership:
servicemesh:
conditions:
- code: NODEPOOL_WORKLOAD_IDENTITY_FEDERATION_REQUIRED
details: One or more node pools have workload identity federation disabled.
documentationLink: https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity
severity: ERROR
controlPlaneManagement:
details:
- code: REVISION_FAILED_PRECONDITION
details: Required in-cluster components are not ready. This will be retried
within 15 minutes.
implementation: TRAFFIC_DIRECTOR
state: FAILED_PRECONDITION
This error is shown if the GKE cluster does not have Workload Identity Federation enabled on all node pools of that cluster, since this is a prerequisites for the installation of Cloud Service Mesh.
To resolve this error message, you must follow instructions to Enable Workload Identity Federation on all node pools. Note that enablement may vary depending on your specific cluster case.
After enablement, the error message should be automatically removed and your
cluster should go back to ACTIVE state. If the issue persists and you need
additional assistance, see Getting support.
MESH_IAM_PERMISSION_DENIED
This error indicates that the Cloud Service Mesh service agent in the fleet project does not have the required permissions on the cluster project. This can happen if the cluster is in a different project than the fleet, or if the permissions were revoked.
To troubleshoot, follow one of the following methods:
Google Cloud console
In the Google Cloud console, navigate to the IAM & Admin > IAM.
Select Include Google-provided role grants.
Review the Principal list.
If you see the service agent with the required IAM role in the list, then it is configured correctly.
If the list doesn't include the service agent and the required role, then continue to the next step.
Grant the Anthos Service Mesh Service Agent role (
roles/anthosservicemesh.serviceAgent) to the Cloud Service Mesh service agent on the project. For instructions, see Manage access to projects, folders, and organizations.
Google Cloud CLI
In the Google Cloud CLI, run the following command to check if the required IAM role has been granted:
gcloud projects get-iam-policy PROJECT_ID \ --flatten="bindings[].members" \ --filter="bindings.members:serviceAccount:service-PROJECT_NUMBER@gcp-sa-servicemesh.iam.gserviceaccount.com AND bindings.role:roles/anthosservicemesh.serviceAgent" \ --format='table(bindings.role)'Review the
ROLElist.If you see any roles in the list, then it is configured correctly.
If you don't see any roles in the list, then the required role was revoked.
To grant the required role to the service agent, run the following command:
gcloud projects add-iam-policy-binding PROJECT_ID \ --member="serviceAccount:service-PROJECT_NUMBER@gcp-sa-servicemesh.iam.gserviceaccount.com" \ --role="roles/anthosservicemesh.serviceAgent"
If permission is verified while issue persists, contact Google customer support for further assistance.
MESH_IAM_CROSS_PROJECT_PERMISSION_DENIED
This error indicates that the fleet project service account does not have enough permission to access a different project (the cluster project or the network project).
For Shared VPC, make sure to grant all the fleet projects service account Anthos Service Mesh Service Agent role in the Shared VPC network project.
For the GKE fleet project and cluster project scenarios, make sure to grant the fleet project service account Anthos Service Mesh Service Agent role in the cluster project.
Example troubleshooting command:
Verify if your network project or cluster project service account has granted
Anthos Service Mesh Service Agentrole to the fleet project service account. If not, run:gcloud projects add-iam-policy-binding NETWORK_OR_CLUSTER_PROJECT_ID \ --member "serviceAccount:service-FLEET_PROJECT_NUMBER@gcp-sa-servicemesh.iam.gserviceaccount.com" \ --role roles/anthosservicemesh.serviceAgentIn addition, confirm that you don't have automation that would remove this binding.
If permission is verified while issue persists, contact Google customer support for further assistance.
CONFIG_VALIDATION_ERROR
This error indicates that some mesh related configuration applied on one or more memberships in the fleet cannot be successfully applied. The error details describe how best the user can address this.
membershipStates:
projects/test-project/locations/us-central1/memberships/my-membership:
servicemesh:
conditions:
- code: CONFIG_VALIDATION_ERROR
details: 'Unsupported ProxyConfig fields: [proxyconfig.statNameLength]'
severity: ERROR
controlPlaneManagement:
Config validation errors for a specific configuration mandate user action. For example, errors regarding usage of unsupported api fields are best addressed by removing the offending configuration from the Mesh API.
In the Cloud Console, navigate to Kubernetes Engine > Secrets & ConfigMaps > istio-asm-managed (the name of the ConfigMap depends upon which release channel you are using; e.g. istio-asm-managed-stable or istio-asm-managed-rapid) then select the YAML tab.
For example, meshConfig.configSources is no longer supported:
apiVersion: v1
data:
mesh: |
enablePrometheusMerge: true
trustDomain: "foobar.svc.id.goog"
trustDomainAliases:
- cluster.local
configSources:
- address: k8s://
- address: fs:///var/lib/istio/config/data