Apache Drill unable to read files from HDFS (Resource error: Failed to create schema tree)

0
votes

After setting up drill on one of the edge nodes of our HDFS cluster, I am unable to read any hdfs files. I can query data from local files (as long as they are in a folder that has 777 permissions) but querying data from hdfs fails with the following error:

Error: RESOURCE ERROR: Failed to create schema tree.

[Error Id: d9f7908c-6c3b-49c0-a11e-71c004d27f46 on server-name:31010] (state=,code=0)

Query:

0: jdbc:drill:zk=local> select * from hdfs.`/names/city.parquet` limit 2;

Querying from local file works fine:

0: jdbc:drill:zk=local> select * from dfs.`/tmp/city.parquet` limit 2;

My HDFS settings are similar to the DFS settings, except for the connection URL being the server address instead of file:///

I can't find anything online regarding this error for drill.

2
hadoopapache-drill
Do you see anything informative in drillbit logs ? You can enable DEBUG level logging using $DRILL_CONF/logback.xml. Also do you have impersonation and authentication enabled ? Please include your drill-override.conf and storage plugin definition as well. - InfamousCoconut

2 Answers

1
votes

"..on one of the edge nodes.."

Drill-bits are meant to run on your data nodes of your Hadoop Cluster as well, not only on edge nodes. This will help for data-locality.

Now, coming to the error you are seeing, It looks like one of your drill-bits may be down or your foreman node(the one you are connected) is having trouble connecting to other nodes.

It will further help if you can give more info like how many drill-bits are in your cluster.

A good starting point should be, to check:

select * from sys.drillbits;

This will tell you which nodes are up/down.

0
votes

In your conf directory of Apache Drill, you need to add the following lines to your drill-override.conf file

drill.exec: {
cluster-id: "drillbits1",
zk.connect: "localhost:2181",
sys.store.provider.local.path="/mypath"
}

Here in place of "/mypath" you need to provide a path of your system where drill will save the storage plugins.

https://community.mapr.com/thread/20905-re-help-with-error-resource-error-failed-to-create-schema-tree You can check the above link.