Can't Access /dbfs/FileStore using shell commands in databricks runtime version 7

0
votes

In databricks runtime version 6.6 I am able to successfully run a shell command like the following:

%sh ls /dbfs/FileStore/tables

However, in runtime version 7, this no longer works. Is there any way to directly access /dbfs/FileStore in runtime version 7? I need to run commands to unzip a parquet zip file in /dbfs/FileStore/tables. This used to work in version 6.6 but databricks new "upgrade" breaks this simple core functionality.

Not sure if this matters but I am using the community edition of databricks.

1
databricksazure-databricksaws-databricks
what does it say the error is? – Smurphy0000

1 Answers

1
votes

WHen you run %sh ls /dbfs/FileStore/tables you can't Access /dbfs/FileStore using shell commands in databricks runtime version 7 because by default, the folder named '/dbfs/FileStore' does not exists in the 'dbfs'.

enter image description here

Try to upload some files in '/dbfs/FileStore/Tables'.

enter image description here

Now, try to run the same command again %sh ls /dbfs/FileStore/tables, now you see the results because we have upload the data into /dbfs/FileStore/tables folder.

enter image description here