import its own ipynb files on google colab

4
votes

I am working with jupyter notebook using google colab(all the files are in the drive). I have 2 files: Exploratory_Data_Analysis.ipynb et PCA.ipynb. I want to import to use the data comming from first one in the secon one. using only the jupyter notebook locally(not with google colaboratory), the importing is working simply by doing this:

!pip install import-ipynb
import import_ipynb
import Exploratory_Data_Analysis as eda

But with google colab I tried the following:

!pip install import-ipynb
import import_ipynb

!pip install -U -q PyDrive
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials
import os
import pandas as pd
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)

listed = drive.ListFile({'q': "'1CXqv7-PZmYrWes4MOk' in 
parents and trashed=false"}).GetList()
for file in listed:
    print('title {}, id {}'.format(file['title'], file['id']))

eda = os.path.join(download_path, 'Exploratory_Data_Analysis.ipynb')
temp_eda = drive.CreateFile({'id': '1YpDhXGeJVtzuxUJS5gKsUbm'})
temp_eda.GetContentFile(eda)

import Exploratory_Data_Analysis

and getting this:

importing Jupyter notebook from Exploratory_Data_Analysis.ipynb
NotJSONError: Notebook does not appear to be JSON: ''...

are there any other ways to import its own ipynb files on google colab?

2
pythonjupyter-notebookgoogle-colaboratory

2 Answers

6
votes

Have you managed to import your notebook/ipynb file to a Google Colab project? The way I migrate my ongoing work in local jupyter-notebook to Google Colab is by using Github and Clouderizer. This method also allows me work on jupyter notebook environment that as if I do it locally but able to sync my work to Google Colab instantly. In addition, this method allows me to import modules .ipynb/.py to my ongoing notebook that I'm working on by simply do something like import <my own python/ipynb module>. I would recommend this setup instead of using hairy linux command line on Google Colab.

Here is the tutorial how to easily setup your notebook from github to Google Colab using Clouderizer: Medium tutorial.

Basically, these are the steps you need to setup your ipynb notebook as well as the dataset folder using Clouderizer:

Clouderizer for Google Colab Project

Prerequisites

  1. Sign Up for google/gmail account
  2. Sign up for Clouderizer account [link]
  3. Create a repository on your github and upload your project (all ipynb, py modules, and even dataset(zip it if < 1GB)) to your repo.

Setup Clouderizer Project

  1. Login to Clouderizer console. On first login, you will be prompted to link your Google Drive with Clouderizer. Follow on-screen instructions to do so. In case it doesn't prompt the link, you can configure your Google Drive by going to Clouderizer dashboard->sidebar menu->Clouderizer Drive. The Clouderizer will setup a folder called 'clouderizer' in your Google Drive to contain your ipynb project (be it Machine Learning or any)
  2. Go back to Clouderizer Dashboard. Then click new Project. As you follow the instruction, you can opt to load the whole github project that contains your ipynb work that you intend to upload to Google Colab.
  3. In step 5, you can include additional dataset you want to work on by specifying the URL for dataset (eg. from kaggle dataset URL). You can also opt and specify if you would like to refactor your project by having separate folder for dataset, main code/modules, and output files.
  4. Next, head to Google Colab/ create a Google Colab file anywhere in you Google Drive, then execute: !wget NS -content disposition 'https://to_whatever_link_you_get_to_console'
  5. Finally, go back to Clouderizer dashboard and check if your project environment is running already and synced to Google Colab. Then click on jupyter notebook icon in that dashboard, just right next to the title name of the project you give. Now, you can start doing your Machine Learning or whatever work you do as if you're doing it on local machine, then sync it immediately at Google Colab.
  6. Finally you will be able to verify if your .ipynb will work and able to import modules/datasets in Google Colab too. Just go back to Google Drive->clouderizer folder->your project. Then try to run and see if everything works as it is in jupyter notebook.
2
votes

Below code worked perfectly for me. 1. Copy all ipynb files into one folder in colab 2. Share the ipynb file from colab, see link: https://www.pingshiuanchua.com/blog/post/importing-your-own-python-module-or-python-file-in-colaboratory 3. Then follow the below steps:

!pip install import-ipynb
import import_ipynb
# Install the PyDrive wrapper & import libraries.
# This only needs to be done once per notebook.
!pip install -U -q PyDrive
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials

# Authenticate and create the PyDrive client.
# This only needs to be done once per notebook.
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)

# Copy the link and remove the front part of the link (i.e. https://drive.google.com/open?id=) to get the file ID.
your_module = drive.CreateFile({'id':'eyetgd1zyxwvutsrqponmlkjihgfedcba'})
your_module.GetContentFile('myfile.ipynb')
import myfile