sqoop export oracle number col RuntimeException: Can't parse input data: '1.0'

Question

I want to export a text table file (input fields terminated by '\t', and I save it to hdfs use pyspark.DataFrame.write.csv('output_history_with_ta_text', sep='\t')) from hdfs to an oracle DB's table. The table has a col named training_accuracy and the type is NUMBER (Just NUMBER, not NUMBER(n,m) ). but I met this problem:

Error: java.io.IOException: Can't export data, please check failed map task logs at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:122) at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: java.lang.RuntimeException: Can't parse input data: '1.0' at PREDICTING_OUTPUT_BASE.__loadFromFields(PREDICTING_OUTPUT_BASE.java:1302) at PREDICTING_OUTPUT_BASE.parse(PREDICTING_OUTPUT_BASE.java:1140) at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:89) ... 10 more Caused by: java.util.NoSuchElementException at java.util.ArrayList$Itr.next(ArrayList.java:834) at PREDICTING_OUTPUT_BASE.__loadFromFields(PREDICTING_OUTPUT_BASE.java:1297) ... 12 more

Problem Here:

Can't parse input data: '1.0'

My Sqoop command is:

sqoop export --connect jdbc:oracle:thin:@192.168.1.3:1521:tmporacle --username admin --password admin --table TMP_TABLE --export-dir /user/hadoop_my/output_history_with_ta_text --input-fields-terminated-by '\t' --verbose --map-column-hive CREATE_DATE=java.sql.Timestamp,MODIFIED_DATE=java.sql.Timestamp,VERSION=java.sql.Date,TRAINING_ACCURACY=Double --map-column-java CREATE_DATE=java.sql.Timestamp,MODIFIED_DATE=java.sql.Timestamp,VERSION=java.sql.Date,TRAINING_ACCURACY=Double

You mentioned csv but you are giving input-fields-terminated-by '\t'. — Sanket

Liang Zulin Liang Zulin · Accepted Answer · 2018-04-24T06:36:24

I found out that the oracle table had one more column which was not matched the hive text table.

Like:

Hive

| col_name | colA | colB |  colD   |
|----------------------------------|
| type     | str  | str  |  double |

Oracle

| col_name | colA | colB | colC               | colD   |
|------------------------------------------------------|
| type     | str  | str  | VARCHAR2(80 BYTE)  | number |

After adding colC in hive table, the problem solved

sqoop export oracle number col RuntimeException: Can't parse input data: '1.0'

1 Answers