Troubleshoot Kafka cluster errors

This page shows you how to resolve issues when creating or configuring a Managed Service for Apache Kafka cluster.

Kafka cluster creation errors

This section lists errors that you might encounter when creating a Kafka cluster.

The service doesn't have permission to access the Cloud KMS key

The following issue occurs when the Managed Service for Apache Kafka service agent is missing the required permission to access the Cloud KMS key.

Service agent service-${PROJECT_NUMBER}@gcp-sa-managedkafka.iam.gserviceaccount.com
has not been granted the required role cloudkms.cryptoKeyEncrypterDecrypter to
encrypt data using the KMS key.`

To resolve this issue, grant the service account the Cloud KMS CryptoKey Encrypter/Decrypter role on the Cloud KMS key. For more information, see required roles for configuring CMEK.

The service doesn't have permission to retrieve the subnet

The following issue occurs when the Managed Service for Apache Kafka service agent is missing the required role to configure networking in the VPC network that the Kafka clients run in.

Service does not have permission to retrieve subnet. Please grant
service-${PROJECT_NUMBER}@gcp-sa-managedkafka.iam.gserviceaccount.com the
managedkafka.serviceAgent role in the IAM policy of the project
${SUBNET_PROJECT} and ensure the Compute Engine API is enabled in project
${SUBNET_PROJECT}`

To resolve this issue, grant the service account the Managed Kafka Service Agent role. For more information, see Connect a cluster across projects.

Networking errors

This section lists errors that you might encounter when configuring networking for a Kafka cluster.

The service failed to set up networking

The following issue occurs if when Managed Service for Apache Kafka can't connect the Kafka cluster to your Virtual Private Cloud subnet.

Managed Service for Apache Kafka failed to set up networking in VPC subnet.

To resolve this issue, perform the following steps:

  • Enable the Compute Engine and Cloud DNS APIs in the parent project of the consumer VPC network.

  • If the Managed Service for Apache Kafka cluster and consumer VPC network are in different projects, configure the required permissions. See Connect a cluster across projects.

  • Ensure that no organization policy constraints prevent the service from creating the necessary resources in the consumer VPC network's project.

For more information, see Configure networking for Managed Service for Apache Kafka.

Client applications can't connect

If your client applications can't connect to your cluster, check for the following issues:

  • Make sure that clients are using the correct bootstrap address.

  • Ensure that the Kafka clients run in a VPC network that is configured to access the Managed Service for Apache Kafka cluster.

  • If you run the Kafka client on a computer or laptop, you can set up a Compute Engine instance to use as a proxy to access to the Managed Service for Apache Kafka cluster. For more information, see Set up a client machine.

Authentication errors

This section lists errors that you might encounter when client applications authenticate to your Kafka cluster.

SASL authentication fails

The following issue occurs when a client can't connect to the cluster by using SASL authentication.

Exception in thread "main" java.util.concurrent.ExecutionException:
org.apache.kafka.common.errors.SaslAuthenticationException:
Authentication failed: Invalid username or password

To resolve the issue, check for the following causes:

  • The password is malformed, and doesn't represent a valid service account key JSON blob when base64 decoded, or a valid access token.

  • The authenticating principal doesn't have the managedkafka.clusters.connect permission on the cluster.

  • The provided username doesn't match the credential's principal.

If a client experiences frequent disconnections every 30 minutes, this can be due to the client not supporting periodic re-authentication. Managed Service for Apache Kafka brokers require clients to re-authenticate every 30 minutes, enforced by the connections.max.reauth.ms broker property. Verify that your Kafka client library version is 2.2.0 or later, and supports re-authentication.

For more information, see Configure SASL authentication.

What's next