R-Studio IDE on Databricks

0
votes

I am trying to import a dataset from my Databricks File System (DBFS) to R-Studio- which is running on Databricks Cluster; and I am facing this issue below.

> sparkDF <- read.df(source = "parquet", path = "/tmp/lrs.parquet", header="true", inferSchema = "true")`

Error: Error in load : java.lang.SecurityException: No token to authorize principal at com.databricks.sql.acl.ReflectionBackedAclClient$$anonfun$com$databricks$sql$acl$ReflectionBackedAclClient$$token$2.apply(ReflectionBackedAclClient.scala:137) at com.databricks.sql.acl.ReflectionBackedAclClient$$anonfun$com$databricks$sql$acl$ReflectionBackedAclClient$$token$2.apply(ReflectionBackedAclClient.scala:137) at scala.Option.getOrElse(Option.scala:121) at com.databricks.sql.acl.ReflectionBackedAclClient.com$databricks$sql$acl$ReflectionBackedAclClient$$token(ReflectionBackedAclClient.scala:137) at com.databricks.sql.acl.ReflectionBackedAclClient$$anonfun$getValidPermissions$1.apply(ReflectionBackedAclClient.scala:86) at com.databricks.sql.acl.ReflectionBackedAclClient$$anonfun$getValidPermissions$1.apply(ReflectionBackedAclClient.scala:81) at com.databricks.sql.acl.ReflectionBackedAclClient.stripReflectionException(ReflectionBackedAclClient.scala:73) at com.databricks.sql.acl.Refle

The DBFS Location is correct, any suggestions or blogs are welcomed for this!

2
rdatabricksazure-databricks
Error suggest missing authentication. That will be specific to your Databricks cluster and R Studio Server settings (please see here: docs.databricks.com/spark/latest/sparkr/rstudio.html) - Dan
Thank you for your Response. I read the link and there are tons of Information mentioned, but nothing related to importing a data set from Azure Data lake or from Databricks File System! - Uttasarga Singh
Ssry, can't provide specific settings for your system, but may be this example helps - this is how I authenticate access for Data Lake Gen 1: spark.conf.set("dfs.adls.oauth2.access.token.provider.type", "ClientCredential") spark.conf.set("dfs.adls.oauth2.client.id", _my client id_) spark.conf.set("dfs.adls.oauth2.credential", _my credentials_) spark.conf.set("dfs.adls.oauth2.refresh.url", _my provider_) How do you authenticate when you're applying operations in Databricks directly? Apply same config to RS configuration. - Dan
While applying Operations on Databricks, I enable a Pass through wile creating the Cluster. Then, I can directly access the data lake by giving the Data-Link in the Command. - Uttasarga Singh

2 Answers

1
votes

The syntax for reading data with R on Databricks depends on whether you are reading into Spark or into R on the driver. See below:

# reading into Spark
sparkDF <- read.df(source = "parquet",
                   path = "dbfs:/tmp/lrs.parquet")


# reading into R
r_df <- read.csv("/dbfs/tmp/lrs.csv")

When reading into Spark, use the dbfs:/ prefix, when reading into R directly use /dbfs/.

0
votes

We should use dbfs before the directory path.

For Example: /dbfs/tmp/lrs.parquet