Spark SQL version of EXEC()

0
votes

Does anyone know of a way in Spark SQL to execute a string variable like the following?

INSERT TableA (Col1,Col2) SELECT Col1,Col2 FROM TableB

I understand that I can obviously write this statement directly. However, I am using a work flow engine where my Insert/Select statement is in String variable. If not, I assume I should use spark_submit. I was looking for other options.

2
apache-sparkapache-spark-sql

2 Answers

0
votes

I'm not sure what environment you're in. If this is a Spark application or a Spark shell you always provide queries as strings:

val query = "INSERT TableA (Col1,Col2) SELECT Col1,Col2 FROM TableB"
sqlContext.sql(query)

(See http://spark.apache.org/docs/latest/sql-programming-guide.html#running-sql-queries-programmatically.)

-1
votes

Spark Sql also support hive queries

insert overwrite table usautomobiles select * from sourcedata

Go Through this link