How to delete folder/files from Databricks mnt directory

0
votes

I am running Databricks Community Edition and I would like to delete files from the following mnt directory

/mnt/driver-daemon/jars

I run the dbutils command:

dbutils.fs.rm('/mnt/driver-daemon/jars/', True)

However, when I run the command I get the following message (which basically means the folder hasn't been deleted)

Out[1]: False

Can someone let me know where I going wrong? Ideally, I would like to delete all the files in the jars folder, however, if someone could just help showing how to delete the folder, that will suffice.

2
databricksazure-databricks
I was able to run dbutils.fs.rm just fine. What happens if you run %fs rm -r /mnt/driver-daemon/jars/? - David Gibson
%fs rm -r /mnt/driver-daemon/jars/ worked. cheers - Carltonp
@DavidGibson Would you like you post that as an answer so OP can mark this question as answered? - CHEEKATLAPRADEEP-MSFT
Hi @DavidGibson, yes I would like to have it posted as an answer. I'm not sure how to do that? - Carltonp
Great, glad to help! And yes I just re-posted as an answer. - David Gibson

2 Answers

1
votes

If you are referring to local /mnt/driver-daemon/jars ,it wouldn't be good idea to remove it since it is symlink to /databricks/jars directory and would impact driver functionality. The path /mnt/driver-daemon/jars/ resolves to dbfs:/mnt/driver-daemon/jars/ whose equivalent local file system directory /dbfs/mnt/driver-daemon/jars. If you want to delete local file system directories, you can prefix file: schema before the path (like file:/tmp/deleteme) with dbutils commands.

 %sh ls -ltrh /dbfs/mnt/driver-daemon/jars /mnt/driver-daemon/jars 
      ls: cannot access '/dbfs/mnt/driver-daemon/jars': No such file or directory
      lrwxrwxrwx 1 root root 16 Jul  2 23:14 /mnt/driver-daemon/jars -> /databricks/jars
1
votes

If dbutils.fs.rm() does not work you can always use the the %fs FileSystem magic commands. To remove a director you can use the following.

%fs rm -r /mnt/driver-daemon/jars/

where

  • %fs magic command to use dbutils
  • rm remove command
  • -r recursive flag to delete a directory and all its contents
  • /mnt/driver-daemon/jars/ path to directory