Unable to load data from Apache hive to ElasticSearch -

Question

I am using CDH5.5,ElasticSearch-2.4.1. I have created Hive table and trying to push the hive table data to ElasticSearch using the below query.

CREATE EXTERNAL TABLE test1_es(
  id string,
  timestamp string, 
  dept string)<br>
ROW FORMAT SERDE 'org.elasticsearch.hadoop.hive.EsSerDe'  
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'  
LOCATION
  'hdfs://quickstart.cloudera:8020/user/cloudera/elasticsearch/test1_es'
TBLPROPERTIES (  'es.nodes'='localhost', 
'es.resource'='sample/test1',
'es.mapping.names' = 'timestamp:@timestamp',
'es.port' = '9200', 
'es.input.json' = 'false', 
'es.write.operation' = 'index', 
'es.index.auto.create' = 'yes'
);<br>
INSERT INTO TABLE default.test1_es select id,timestamp,dept from test1_hive;

I'm getting the below error in the Job Tracker URL "

 Failed while trying to construct the redirect url to the log server. Log Server url may not be configured. <br>
java.lang.Exception: Unknown container. Container either has not started or has already completed or doesn't belong to this node at all. "

It will throw "FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask" in hive terminal.

I tried all the steps mentioned in forums like including /usr/lib/hive/bin/elasticsearch-hadoop-2.0.2.jar in hive-site.xml, adding ES-hadoop jar to HIVEAUXJARS_PATH, copied yarn jar to /usr/lib/hadoop/elasticsearch-yarn-2.1.0.Beta3.jar also. Please suggest me how to fix the error.

Thanks in Advance, Sreenath

Paul Lam Paul Lam · Accepted Answer · 2017-02-03T02:49:55

I'm dealing with the same problem, and I found the execution error thrown by hive is caused by a timestamp field of string type which could not be parsed. I'm wondering whether timestamp fields of string type could be properly mapped to es, and if not this could be the root cause.

BTW, you should go to the hadoop MR log to find more details about the error.

Unable to load data from Apache hive to ElasticSearch -

2 Answers