DEV Community

JJ Dev
JJ Dev

Posted on

How do I troubleshoot issues about Amazon MSK cluster?

1. Create Topic use SASL/SCRAM

When I ran this command to create a topic. These are some errors below.

bin/kafka-topics.sh --create --bootstrap-server <endpoint> --topic <topic_name> --command-config client.properties
Enter fullscreen mode Exit fullscreen mode

[java.lang.OutOfMemoryError Java heap space]
[ERROR org.apache.kafka.common.errors.TimeoutException: The AdminClient thread has exited. Call: createTopics]

Image description

Soulation
The error OutOfMemoryError occurred, which is a common issue when connecting to Kafka using SASL_SSL. This is not actually related to memory, but rather because the client was not configured to connect using SSL.
You should configure your admin client with --command-config <ssl.conf>


2. Public Access by SASL/SCRAM

If you are using the SASL/SCRAM or mTLS access-control methods, and want to enable public access msk. You will show the error.
Image description

Soulation
After you set the Apache Kafka ACLs for your cluster, update the cluster's configuration.
Add Cluster configuration allow.everyone.if.no.acl.found=false
Image description

3. Consumers ACL rules

When you config allow.everyone.if.no.acl.found=false, must be set for specific consumer groups, and --group '*' cannot be used.

[org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: console-consumer-1736]
Image description

Soulation
Now, explicitly specify that user:user_name can read the topic:demo-topic and use group:test.

bin/kafka-acls.sh --command-config client_sasl.properties --bootstrap-server <endpoint> --add --allow-principal "User:user_name" --operation Read --group=test --topic demo-topic
Enter fullscreen mode Exit fullscreen mode

Top comments (0)