import local file to google colab

Question

I don't understand how colab works with directories, I created a notebook, and colab put it in /Google Drive/Colab Notebooks.

Now I need to import a file (data.py) where I have a bunch of functions I need. Intuition tells me to put the file in that same directory and import it with:

import data

but apparently that's not the way...

I also tried adding the directory to the set of paths but I am specifying the directory incorrectly..

Can anyone help with this?

Thanks in advance!

Answer 1

Now google is officially providing support for accessing and working with Gdrive at ease.

You can use the below code to mount your drive to Colab:

from google.colab import drive
drive.mount('/gdrive')
%cd /gdrive/My\ Drive/{location you want to move}

Answer 2

Colab notebooks are stored on Google Drive. But it is run on another virtual machine. So, you need to copy your data.py there too. Do this to upload data.py through Colab.

from google.colab import files
files.upload()
# choose the file on your computer to upload it then
import data

Answer 3

To easily upload a local file you can use the new Google Colab feature:

• click on right arrow on the left of your screen (below the Google Colab logo)
• select Files tab
• click Upload button

It will open a popup to choose file to upload from your local filesystem.

Answer 4

To upload Local files from system to collab storage/directory.

from google.colab import files
def getLocalFiles():
    _files = files.upload()
    if len(_files) >0:
       for k,v in _files.items():
         open(k,'wb').write(v)
getLocalFiles()

Answer 5

So, here is how I finally solved this. I have to point out however, that in my case I had to work with several files and proprietary modules that were changing all the time.

The best solution I found to do this was to use a FUSE wrapper to "link" colab to my google account. I used this particular tool:

https://github.com/astrada/google-drive-ocamlfuse

There is an example of how to set up your environment there, but here is how I did it:

# Install a Drive FUSE wrapper.
!apt-get install -y -qq software-properties-common python-software-properties module-init-tools
!add-apt-repository -y ppa:alessandro-strada/ppa 2>&1 > /dev/null
!apt-get update -qq 2>&1 > /dev/null
!apt-get -y install -qq google-drive-ocamlfuse fuse


# Generate auth tokens for Colab
from google.colab import auth
auth.authenticate_user()
# Generate creds for the Drive FUSE library.
from oauth2client.client import GoogleCredentials
creds = GoogleCredentials.get_application_default()
import getpass
!google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null 2>&1 | grep URL
vcode = getpass.getpass()
!echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret}

At this point you'll have installed the wrapper and the code above will generate a couple of links for you to authorize access to your google drive account.

The you have to create a folder in the colab file system (remember this is not persistent, as far as I know...) and mount your drive there:

# Create a directory and mount Google Drive using that directory.
!mkdir -p drive
!google-drive-ocamlfuse drive

print ('Files in Drive:')
!ls drive/

the !ls command will print the directory contents so you can check it works, and that's it. You now have all the files you need and you can make changes to them with no further complications. Remember that you may need to restar the kernel to update the imports and variables.

Hope this works for someone!

Answer 6

you can write following commands in colab to mount the drive

from google.colab import drive 
drive.mount('/content/gdrive') 

and you can download from some external url into the drive through simple linux command wget like this

!wget 'https://dataverse.harvard.edu/dataset'