We are currently using Confluent HDFS Sink Connector platform within docker container to write data from Kafka(separate Kafka cluster) to HDFS(separate Hadoop cluster). By default the connector platform writes data to HDFS with root user and wheel group. How can i configure connector to use a specific hadoop user/group ? Is there an environment variable I need to set in docker ? Thanks.
1 Answers
0
votes
The Java process in the Docker container runs as root.
You need to either make your own container with your own user account or run the Connect Workers as a different Unix account in some other way.
You could try setting HADOOP_IDENT_USER or HADOOP_USER_NAME environment variables, but I think these are only pulled by the Hadoop scripts, not the Java API
Keep in mind that user accounts in Hadoop don't really matter if you're not using a Kerberized cluster
