Connected to snowflake using python JDBC driver but not able to do that with pyspark in jupyter notebook?Already confirmed correctness of my username and password.
Environment details :-
Referring this tutorial
https://docs.snowflake.net/manuals/user-guide/spark-connector-use.html#label-spark-options.
Error stack:-
Any Leads?
Just to contribute to community posting answer
Following the version of spark and driver solved the problem: below combination of jars and spark worked for me. Spark 2.3.3 (spark-2.3.3-bin-hadoop2.7) Python 3.6.6(can be 3.7) snowflake-jdbc-3.6.12.jar spark-snowflake_2.11-2.4.8.jar snowflake server:-(3.27.2)
If you come across below error
Py4JJavaError: An error occurred while calling o272.load.
: java.sql.SQLClientInfoException: The only allowable value is
application name.
hodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Remove sfAccount property from config sfOptions object official snowflake document
sfOptions = {
"sfURL" : "<account_name>.snowflakecomputing.com",
"sfAccount" : "<account_name>",
"sfRole" : "<user_role_name>",
"sfUser" : "<user_name>",
"sfPassword" : "<password>",
"sfDatabase" : "<database>",
"sfSchema" : "<schema>",
"sfWarehouse" : "<warehouse>",
}
Error thrown is pretty much generalized which further create confusion what was wrong or missed anything to add extra.
to add to the community post and the correct answer, this also works with:
and the following jars:
but you need to remove "sfAccount" from snowflake documentation for sfOptions
var sfOptions = Map("sfURL" -> "xxx.snowflakecomputing.com","sfUser" -> "your_sf_user","sfPassword" -> "your_sf_password","sfDatabase" -> "your_sf_database","sfSchema" -> "your_sf_schema","sfWarehouse" -> "your_sf_warehouse")`
