Unable to start Elasticsearch Sink

0
votes

I'm trying to get my elasticsearch sink up and running. However I'm getting the following error and am running out of ideas on how to fix it. Any help is appreciated. Here is the error:

2015-02-10 11:43:31,234 ERROR org.apache.flume.lifecycle.LifecycleSupervisor: Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@79a805d2 counterGroup:{ name:null counters:{} } } - Exception follows. java.lang.NoSuchFieldError: LUCENE_3_6 at org.elasticsearch.Version.(Version.java:45) at org.elasticsearch.client.transport.TransportClient.(TransportClient.java:168) at org.elasticsearch.client.transport.TransportClient.(TransportClient.java:123) at org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.openClient(ElasticSearchTransportClient.java:201) at org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.(ElasticSearchTransportClient.java:79) at org.apache.flume.sink.elasticsearch.client.ElasticSearchClientFactory.getClient(ElasticSearchClientFactory.java:48) at org.apache.flume.sink.elasticsearch.ElasticSearchSink.start(ElasticSearchSink.java:357) at org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46) at org.apache.flume.SinkRunner.start(SinkRunner.java:79) at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)

I am running CDH 5.3 and elasticsearch 1.4.2

2
elasticsearchflume-ng
Probably more information like the flume config would be helpful. It'll help viewers understand your problem and you may even fix it yourself when you're forced to explain it here. – Felipe

2 Answers

0
votes

I suspect that you dropped your elastic search jars into FLUME_HOME/lib

The problem here is due to conflicting versions of Elastic Search in Flume. The libraries you provided aren't the same as the ones used in flume. Flume 1.5 is linked against Elastic Search 4.9.

0
votes

Ok, figured it out after you guys pointed me in the right direction.

Since I'm using CHD 5.3, I needed to copy the elasticsearch 1.4.2 jar into the /user/lib/hadoop dir and not the flume-ng dir. After doing this it worked perfectly.

Thanks everyone for the help!