Kafka unable to produce/consume message

0
votes

I've got a Kerberos SASL_SSL enabled Kafka authentication and the default kafka.security.authorizer.AclAuthorizer authorization setup in place. All the necessary ACLs (which i believe) is there in place however when i try to produce a message it results with the following error [2021-07-30 09:19:33,397] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 3 : {camtmodified.v1=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient) [2021-07-30 09:19:33,399] ERROR [Producer clientId=console-producer] Topic authorization failed for topics [camtmodified.v1] (org.apache.kafka.clients.Metadata) [2021-07-30 09:19:33,400] ERROR Error when sending message to topic camtmodified.v1 with key: null, value: 8 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [camtmodified.v1]

Following is the ACL list i have in place which i believe is what is sufficient, as this is the same list of ACLs i have enabled for few other principals which seem to be work fine but not just this one "user:credusr"

Current ACLs for resource ResourcePattern(resourceType=TOPIC, name=camtmodified.v1, patternType=LITERAL): (principal=cluster:kafka-cluster, host=, operation=WRITE, permissionType=ALLOW) (principal=Group:, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=cluster:kafka-cluster, host=, operation=READ, permissionType=ALLOW) (principal=user:credusr, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=user:credusr, host=, operation=READ, permissionType=ALLOW) (principal=Group:, host=, operation=READ, permissionType=ALLOW) (principal=cluster:kafka-cluster, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=user:credusr, host=, operation=CREATE, permissionType=ALLOW) (principal=cluster:kafka-cluster, host=, operation=CREATE, permissionType=ALLOW) (principal=user:credusr, host=, operation=WRITE, permissionType=ALLOW)

Current ACLs for resource ResourcePattern(resourceType=CLUSTER, name=kafka-cluster, patternType=LITERAL): (principal=user:credusr, host=, operation=ALL, permissionType=ALLOW) (principal=cluster:kafka-cluster, host=, operation=ALL, permissionType=ALLOW) (principal=Cluster:kafka-cluster, host=*, operation=IDEMPOTENT_WRITE, permissionType=ALLOW)

Current ACLs for resource ResourcePattern(resourceType=GROUP, name=*, patternType=LITERAL): (principal=User:credusr, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=Group:, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=cluster:kafka-cluster, host=, operation=READ, permissionType=ALLOW) (principal=user:credusr, host=, operation=READ, permissionType=ALLOW) (principal=User:credusr, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=Group:, host=, operation=READ, permissionType=ALLOW) (principal=group:, host=, operation=READ, permissionType=ALLOW)

I did enable the DEBUG mode for kafka authorization logs and see the following entries. As the DESCRIBE operation is explicity allowed as shown in the ACL list - i'm not sure why it's complaining the DESCRIBE operation as denied, unless the log message is quite misleading!
[2021-07-29 13:40:49,750] INFO Principal = User:credusr is Denied Operation = Describe from host = 10.X.X.X on resource = Topic:LITERAL:camtmodified.v1 for request = Metadata with resourceRefCount = 1 (kafka.authorizer.logger) [2021-07-29 13:40:49,751] INFO Principal = User:credusr is Denied Operation = IdempotentWrite from host = 10.X.X.X on resource = Cluster:LITERAL:kafka-cluster for request = InitProducerId with resourceRefCount = 1 (kafka.authorizer.logger) [2021-07-29 13:40:50,678] INFO Principal = User:credusr is Denied Operation = Describe from host = 10.X.X.X on resource = Topic:LITERAL:camtmodified.v1 for request = Metadata with resourceRefCount = 1 (kafka.authorizer.logger)

Any suggestions will be much appreciated.

Thanks

1
apache-kafkakafka-consumer-api

1 Answers

0
votes

Got this resolved finally!!! Just posting the resolution here for those who might end up with a similar issue and looking for a resolution. Basically the issue was with the case sensitivity of the Kafka ACL commands. Even though the syntax and resource parameters were correct, the ACL commands were entered with a lower case causing the authorization not being honoured and eventually being denied. In the above case - my principal name as entered as "user" as opposed to the "User" format and thereby causing all the corresponding authorizations to fail! In summary: Please be mindful on the ACL commands with appropriate case sensitivity as it seems like the commands are indeed case sensitive. Although it was a minor probem , i'd to find it in a hard way and posting the answers just in case if this helps someone as a beginner (like me) in near future!

Cheers,