Apache Drill with Kerberos

2
votes

Does anyone know how to enable kerberos with Apache Drill? Is it possible. I can't seem to find any documentation on it, or any questions/answers floating around with the information on it. I am currently running a CDH cluster.

I am getting this error when trying to use HDFS with Drill:

Error: PERMISSION ERROR: SIMPLE authentication is not enabled.  
Available:[TOKEN, KERBEROS]
3
apache-drill

3 Answers

2
votes

HDFS + Kerberos integration isn't currently supported / tested / documented. Vote on this ticket to track when it becomes available:

https://issues.apache.org/jira/browse/DRILL-3584

0
votes

There isn't any documentation that the Drill team provides about how to enable kerberos and they haven't tested kerberos with Drill. Drill Eng. does believe that it should work.

0
votes
In order to gain access onto the cluster once Kerberized, you must configure certain files in order to gain access.
Make an HDFS Superuser account as indicated in this Cloudera doc. On the Main Node, run 

•sudo kadmin.local 
In addition, add an 'hdfs' principal with this command

•addprinc hdfs@LOCALDOMAIN   -- Where localdomain is the principal name

In order to enable authentication with Kerberos, we also need to copy the file hadoop-yarn-api.jar into Drill's class path. Example given below

•cp /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/client/hadoop-yarn-api.jar ~/apache-drill/jars/

The above step and the three following must be performed on each node of the cluster that an Apache Drill is installed.

Next, Drill's conf/core-site.xml file should be edited to contain the following snippet of xml. You might have to copy this file from /etc/hadoop/conf.cloudera.yarn/core-site.xml, etc or a similar path. 

<property>
  <name>hadoop.security.authentication</name>
  <value>kerberos</value>
</property>

After this step, you will also need to add the following xml snippet below to the drill core-site.xml file. In this instance, hdfs/_HOST@LOCALDOMAIN is my principal property. The property can be found on the hdfs-site.xml file

<property>
  <name>dfs.namenode.kerberos.principal</name>
  <value>hdfs/_HOST@LOCALDOMAIN</value>
</property>

All that is left to do is create an 'hdfs' Kerberos ticket for the user that we're logged into

•kinit hdfs   -- hdfs is the super user
Then start up each of the drillbits

•/opt/apachedrillfolder/bin/Drillbit.sh start
So now, Drill has both the configuration and the authority to use our kerberized HDFS store. Give it a shot by opening up a Drill prompt (drill-conf) and trying a query