inferschema issue with com.databricks.spark.csv

0
votes

I have fallowing csv

Name,Department
Sam,Software
1,Data Analytics

the 'name' column has one line with value 'Sam' and another line with value 1 'Sam' is String and 1 is Int

using com.databricks.spark.csv i cant load both lines to dataframe

what can i do to solve this problem

thanks

1
scalaapache-sparkdatabricks
The column should have the same schema. - Lamanus

1 Answers

0
votes

inferSchema option is used to identify the datatype of the columns, And if your column data has one row as string and one row as int it will fail obviously.

And if you still want to insert the same data try this

Name,Department
Sam,Software
'1',Data Analytics

Hope this will solve ur problem