In one of our project, we are using Azure SQL DB, authentication mode is Access Token based. We are using Spark Java to connect. We are able to connect to DB when we run our spark job as standalone but when we are running in YARN mode, we are not able to connect. We are using the library “azure-activedirectory-library-for-java”
We are getting the error as: Login failed for user: ‘’. ClientConnectionId:
Exception: INFO Client: client token: N/A diagnostics: User class threw exception: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user ''. ClientConnectionId:2b288e9e-15cc-448a-8210-0d184d27a12b ApplicationMaster host: 172.25.3.7 ApplicationMaster RPC port: 0 queue: default start time: 1538583454649 final status: FAILED tracking URL:
References:
https://github.com/Azure/azure-sqldb-spark
https://docs.microsoft.com/en-us/azure/sql-database/sql-database-spark-connector
Thanks in advance.
Thanks & Regards,
-Venkat
Below is the exception stacktrace:
18/10/08 10:07:16 WARN SparkSession$Builder: Using an existing SparkSession; some configuration may not take effect. 18/10/08 10:07:16 ERROR ApplicationMaster: User class threw exception: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user ''. ClientConnectionId:8ecbc2ae-cede-4b1d-8eba-f362010f71e6 com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user ''. ClientConnectionId:8ecbc2ae-cede-4b1d-8eba-f362010f71e6 at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:217) at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:251) at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:81) at com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:3077) at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:2360) at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$100(SQLServerConnection.java:43) at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:2346) at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:6276) at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1793) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1404) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:1068) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:904) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:451) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1014) at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:61) at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:52) at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:58) at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation.(JDBCRelation.scala:114) at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:52) at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:309) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:178) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:146) at org.apache.spark.sql.DataFrameReader.jdbc(DataFrameReader.scala:193) at com.microsoft.azure.sqldb.spark.connect.DataFrameReaderFunctions.sqlDB(DataFrameReaderFunctions.scala:44) at com.sew.data.SQLDBConnectionEngine$.main(SQLDBConnectionEngine.scala:71) at com.sew.data.SQLDBConnectionEngine.main(SQLDBConnectionEngine.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:646) 18/10/08 10:07:16 INFO ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user ''. ClientConnectionId:8ecbc2ae-cede-4b1d-8eba-f362010f71e6) 18/10/08 10:07:16 INFO SparkContext: Invoking stop() from shutdown hook 18/10/08 10:07:16 INFO AbstractConnector: Stopped Spark@683861d4{HTTP/1.1,[http/1.1]}{0.0.0.0:0}
