DSE with Hadoop: Error in getting started

Question

I am facing a problem in DSE with Hadoop.

Let me describe the setup, including steps in some details, for you to be able to help me.

I set up a three-node cluster of DSE, with the cluster name as 'training'. All three machines are running Ubuntu 14.04, 64-bit, 4 GB RAM.

DSE was installed using the GUI installer (sudo command). After the installation, cassandra.yaml file was modified for

rpc_address = 0.0.0.0

One-by-one the three nodes were started. A keyspace with replication_factor = 3 was created. Data inserted and accessed from any other node, successfully.

Then DSE installed on a fourth machine (let's call this machine as HadoopMachine), again, with the same configuration, using GUI installer (sudo).

/etc/default/dse modified as follows:

HADOOP_ENABLED = 1

Then, on this HadoopMachine, the following command is run:

sudo service dse start

So far so good.

Then from the installation directory:

bin/dse hadoop fs -mkdir /user/hadoop/wordcount

This fails. Gives a very long series of error messages, running into hundreds of lines, ending with:

    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
    at com.datastax.bdp.loader.SystemClassLoader.tryLoadClassInBackground(SystemClassLoader.java:163)
    at com.datastax.bdp.loader.SystemClassLoader.loadClass(SystemClassLoader.java:117)
    at com.datastax.bdp.loader.SystemClassLoader.loadClass(SystemClassLoader.java:81)
    at com.datastax.bdp.loader.SystemClassLoader.loadClass(SystemClassLoader.java:75)
    at java.util.ResourceBundle$RBClassLoader.loadClass(ResourceBundle.java:503)
    at java.util.ResourceBundle$Control.newBundle(ResourceBundle.java:2640)
    at java.util.ResourceBundle.loadBundle(ResourceBundle.java:1501)
    at java.util.ResourceBundle.findBundle(ResourceBundle.java:1465)
    at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419)
    at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1361)
    at java.util.ResourceBundle.getBundle(ResourceBundle.java:890)
    at sun.util.resources.LocaleData$1.run(LocaleData.java:164)
    at sun.util.resources.LocaleData$1.run(LocaleData.java:160)
FATAL ERROR in native method: processing of -javaagent failed
bin/dse: line 192: 12714 Aborted                 (core dumped) "$HADOOP_BIN/hadoop" "$HADOOP_CMD" $HADOOP_CREDENTIALS "${@:2}"

I don't know what the problem is, and how to fix it.

Will appreciate any help. Thanks.

Try dse hadoop fs -mkdir /user/hadoop/wordcount/input. If that doesn't work try sudo dse hadoop fs -mkdir /user/hadoop/wordcount/input. The cd install_location and bin/dse hadoop fs command instruction is for the tarball or no services installation. — catpaws
@catpaws, tried both these. Still the same outcome. Giving the command "dse hadoop" displays the list of commands, which suggests that hadoop is running and accessible; but the command "dse hadoop fs" or "dse hadoop fs -mkdir /user/hadoop/wordcount" results in the above error. — Pradyumn
Can you connect to cqlsh on the HadoopMachine? What is the output of desc keyspace cfs; Do you have the HadoopMachine in a separate data center? What is the output of nodetool status? — catpaws
Did you enable password protection? If so, please see datastax.com/documentation/datastax_enterprise/4.5/… — catpaws
@catpaws: No, not connecting to cqlsh also ("could not connection to localhost:9160"). I must be making some fundamental, silly mistake, but unable to pinpoint it. Confused: what to look for and where. — Pradyumn

Pradyumn Pradyumn · Accepted Answer · 2014-11-29T14:09:01

I managed to find the solution, after a lot of struggle. I had been guessing all this time, that the problem would be one mis-step somewhere that was not very obvious, at least to me, and that's how it turned out.

So for the benefit of anybody else who may face the same problem, what the problem was and what worked is as follows.

DSE documentation specifies that for DSE with integrated Hadoop you must have Oracle JRE 7. I, perhaps foolishly, assumed it would mean Oracle JRE 7 or higher. So I had JRE 8 on my machine, and never realized that that would be the issue. When I removed JRE 8 and installed JRE 7, and bingo, it worked.

I am amazed. Now I realize that since DSE uses Hadoop 1.0.4 (an ancient version), it works with JRE 7 only. JRE 8 must have come after Hadoop 1.0.4 and something in JRE 8 must be incompatible with JRE 7, I guess.

DSE with Hadoop: Error in getting started

1 Answers