Sqoop 1.99.7 Installation on Centos 7

Question

I am new to Hadoop Eco System.

I am in the process of setting up a Hadoop Eco System on a single node. Hadoop system is already been setup and is working fine along with HBASE and HIVE. To to connect to external sources of data, I am setting up sqoop 1.99.7 server. I am finding it difficult to setup, as documentation is creating sort of confusion. Can some one help me with step by step process... The steps done by me so far ..

Java 1.7 version - available
Hadoop common - already been setup and verified
Downloaded and installed sqoop version 1.99.7
configured the ~/.bashrc file with export SQOOP_HOME=/usr/lib/sqoop export PATH=$PATH:$SQOOP_HOME/bin
Source the bashrc file

Now I am not clear, as per documentation provided, on and I am stuck here for the last one day... 1. Creating Third party jars - I have created a directory /var/lib/sqoop2 using mkdir -p /var/lib/sqoop2

(Creation did not give any error and unable to locate this directory)
Hence copy all mysql-jdbc* files, postgresql-jdbc could not be done. Also
Unable to locate mysl-jdbc* files in the server path

2. I presume no action is required to configure the sqoop_boot starp. properties and sqoop.properties

Unless the above steps completed, I ca not start the sqoop server. Can one correct me on the above as well what needs to be done.. Thanks in advance.. Sreeram

Sreeram Vijay Maddury Sreeram Vijay Maddury · Accepted Answer · 2017-08-20T08:44:36

Just to add further, when I start the server, i get the following output and the sqoop server does not start. what is remedy?

[root@localhost sqoop]# ./bin/sqoop.sh server start
Setting conf dir: /home/sreeram/sqoop/conf
Sqoop home directory: /home/sreeram/sqoop
Starting the Sqoop2 server...
2017-08-20 14:06:16,511 INFO  [main] core.SqoopServer (SqoopServer.java:initialize(55)) - Initializing Sqoop server.
2017-08-20 14:06:16,521 INFO  [main] core.PropertiesConfigurationProvider (PropertiesConfigurationProvider.java:initialize(99)) - Starting config file poller thread
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/sreeram/sqoop/server/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/sreeram/apache-hive-2.3.0-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
Exception in thread "main" java.lang.RuntimeException: Failure in server initialization
    at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:68)
    at org.apache.sqoop.server.SqoopJettyServer.<init>(SqoopJettyServer.java:67)
    at org.apache.sqoop.server.SqoopJettyServer.main(SqoopJettyServer.java:177)
Caused by: org.apache.sqoop.common.SqoopException: MAPREDUCE_0002:Failure on submission engine initialization - Invalid Hadoop configuration directory (not a directory or permission issues): /etc/hadoop/conf/
    at org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.initialize(MapreduceSubmissionEngine.java:97)
    at org.apache.sqoop.driver.JobManager.initialize(JobManager.java:257)
    at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:64)
    ... 2 more

Sqoop 1.99.7 Installation on Centos 7

1 Answers