Importing data using sqoop in HDFS beugs

Question

I am following this tutorial http://hadooped.blogspot.fr/2013/05/apache-sqoop-for-data-integration.html. And I have installed hadoop services (hdfs, hive, sqoop, hue, ...) using cloudera manager. I am using Ubuntu 12.04 TLS. When trying to import data from Mysql to HDFS, mapreduce jobs takes infinite time without returning any error. Knowing that the imported table has 4 columns and 10 rows.

this is what I do:

    sqoop import --connect jdbc:mysql://localhost/employees --username hadoop --password password --table departments -m 1 --target-dir /user/sqoop2/sqoop-mysql/department

    Warning: /opt/cloudera/parcels/CDH-5.5.2-1.cdh5.5.2.p0.4/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
    Please set $ACCUMULO_HOME to the root of your Accumulo installation.
    16/02/23 17:49:09 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.5.2
    16/02/23 17:49:09 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
    16/02/23 17:49:10 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
    16/02/23 17:49:10 INFO tool.CodeGenTool: Beginning code generation
    16/02/23 17:49:11 INFO manager.SqlManager: Executing SQL statement:  SELECT t.* FROM `departments` AS t LIMIT 1
    16/02/23 17:49:11 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `departments` AS t LIMIT 1
    16/02/23 17:49:11 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce
    Note: /tmp/sqoop-root/compile/6bdeb198a0c249392703e3fc0070cb64/departments.java uses or overrides a deprecated API.
    Note: Recompile with -Xlint:deprecation for details.
    16/02/23 17:49:19 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/6bdeb198a0c249392703e3fc0070cb64/departments.jar
    16/02/23 17:49:19 WARN manager.MySQLManager: It looks like you are importing from mysql.
    16/02/23 17:49:19 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
    16/02/23 17:49:19 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
    16/02/23 17:49:19 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
    16/02/23 17:49:19 INFO mapreduce.ImportJobBase: Beginning import of departments
    16/02/23 17:49:20 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
    16/02/23 17:49:24 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
    16/02/23 17:49:24 INFO client.RMProxy: Connecting to ResourceManager at hadoopUser/10.0.2.15:8032
    16/02/23 17:49:31 INFO db.DBInputFormat: Using read commited transaction isolation
    16/02/23 17:49:31 INFO mapreduce.JobSubmitter: number of splits:1
    16/02/23 17:49:33 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1456236806433_0004
    16/02/23 17:49:34 INFO impl.YarnClientImpl: Submitted application application_1456236806433_0004
    16/02/23 17:49:34 INFO mapreduce.Job: The url to track the job: http://hadoopUser:8088/proxy/application_1456236806433_0004/
    16/02/23 17:49:34 INFO mapreduce.Job: Running job: job_1456236806433_0004

Job_image

regards,

The logs you have posted has a section The url to track the job: you can look the url up next time you run the job to check the Map Reduce Logs. Looks like your connection is going through as INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM departments` AS t LIMIT 1` was executed successfully. — user 923227
Hi Sumit Kumar Ghosh, I have opened the link. and I noticed that the progress is 0%. I do not understand what's the problem, because I don't have any error message. is it a permission issue ? I have executed the command as hdfs this time and same result. I have added the image of the link above. — manie

user 923227 user 923227 · Accepted Answer · 2016-02-28T22:33:41

The MapReduce Job is not getting Launched. You need to run a test wordcount job on the cluster.

Importing data using sqoop in HDFS beugs

1 Answers