Running IPython Notebook over Python3 in a VM

Question

I'm struggling to put together a simple VM that I can use to run an IPython notebook over Python3.

I figured the best way was to set up a virtual env for Python and then install the required libraries into that before starting the IPython server, although maybe a better way would be to separate out the virtualenv creation into a shell script?

UPDATE: One major problem I had was with incomplete paths...

##Vagrantfile

Vagrant.configure("2") do |config|

  config.vm.box = "precise64"
  config.vm.box_url = "http://files.vagrantup.com/precise64.box"

  config.ssh.forward_x11 = true

  config.vm.provision :puppet, 
    :options => "--modulepath=/vagrant/modules" do |puppet|
    puppet.manifests_path = "."
    puppet.manifest_file = "site.pp"
  end

  config.vm.define :python3 do |python3|
    python3.vm.hostname = "python3"
    python3.vm.provider :virtualbox do |virtualbox|
      virtualbox.name = "python3"
    end
    python3.vm.network :forwarded_port, guest: 8888, host: 8888
  end
end

###site.pp
node default {
  include testVM
}


#in modules/testVM//manifests/box/python3.pp
class testVM::box::python3 {
  #I was hoping this require statement would install python3 & virtualenv first?
  require python3::base

  #But it doesn't seem to because the following exec runs first and fails?
  #ssh'ing into the VM and the base packages didn't install?
  exec {
    'py3-venv':
      command => '/usr/bin/virtualenv --python=/usr/bin/python3 testpy3',
      require => Package['openssh-server','python-virtualenv'];
  ##I think problem is narrowed down to this - how do I run source or . ?
    'py3-activate':
      command => 'source testpy3/bin/activate',
      require=>Exec['py3-venv'];
  }

  #Try to add in a dependency to force package install - still doesn't work?
  Package['python3']->Exec['py3-venv'] 

  #Install in a small package to check it appears in python3 env
  package {
    [
      'ipythonblocks'
    ]: ensure   => latest,
       provider => 'pip';
  }
}

#I assumed this would install before the exec fired but it doesn't seem to?
class testVM::box::python3::base {
  package { 'python3':
    ensure => latest
  }
  package {
    [ 'ipython3',
      'python-virtualenv'
    ]: require => Package['python3'];
  }
}

If I ssh in and install everything by hand, I can run the source command fine from the commandline.

However, if I try to reprovision the machine with the stuff installed by hand I still get an error?

'source testpy3/bin/activate' is not qualified and no path was specified. Please qualify the command or specify a path.

SOLUTION TO THAT PROBLEM: the solution to the problem of source not being qualified can be found here: https://ask.puppetlabs.com/question/2177/how-to-execute-source-command/ The solution is: command => "/bin/bash -c 'source testpy3/bin/activate'"

BUT A NEW PROBLEM ARISES: err: /Stage[main]/Infinite_interns::Box::Python3/Exec[py3-venv]/returns: change from notrun to 0 failed: /usr/bin/virtualenv –-no-site-packages --python=/usr/bin/python3 testpy3 returned 1 instead of one of [0] at /vagrant/modules/infinite_interns/manifests/box/python3.pp:14

DOH - in my actual script I had /usr/bin/virtualenv –-no-site-packages --python=/usr/bin/python3 testpy3, it should have just been /usr/bin/virtualenv --python=/usr/bin/python3 testpy3`

So that's all good now...

UPDATE 2 - or not.. I added Exec['py3-venv']->Package['ipythonblocks'] to make sure the virtualenv stuff was installed and executed before trying to install the additional module via pip, but whilst it executes I can't see the installed library in the virtualenv in the VM?

So the question now is: can puppet pip install libraries into the virtualenv testpy3?m Or would I be better not using the virtualenv and just calling python3 and related tools directly?

ANSWER: I have given up on virtualenv and am settling for 'just' running python3 in global scope. I'm on ubuntu, so need to easy_install3 pip as described in https://askubuntu.com/questions/412178/how-to-install-pip-for-python-3-in-ubuntu-12-04-lts ; to use pip3 as the provider, https://github.com/bodepd/puppet-pip/blob/master/lib/puppet/provider/package/pip3.rb

Onto the PS bit to see if that works..

PS The next step will be to get the notebook running via a file in etc/init - would the following do it? (I haven't got that far yet!) Is this likely to work or does IPython notebook under python 3 use a different starting incantation?!

script
  export HOME="/root"
  /vagrant/notebooks/ipython3 notebook --ip 0.0.0.0
end script

Have you tried the full path, so something like source /usr/bin/python3/testpy3/bin/activate ? — Andy Hayden
@AndyHayden Hmm... nope - that doesn't seem to work? Is that where the testpy3 directory would be created anyway? (I tried various absolute paths on the files to same effect?) — psychemedia
I'm not sure where it'll be created, but perhaps if you can locate it you can use the full path. First thought it might be in home folder somewhere but can't recall... — Andy Hayden
@filmor We are packaging a VM up for use by distance education students - preinstalled with various libraries and databases, so they can use any platform, cloud VM etc — psychemedia

rachel rachel · Accepted Answer · 2014-08-28T18:58:28

With the caveat that I haven't quite gotten it to work for me reliably (that is, it's worked a few times & then stopped, some reinstalls work & some don't [seems to be affected by other packages you would install after, a non-clean DST install, etc]), building your VM based on the Data Science Toolbox, found here: http://datasciencetoolbox.org/ . In our office we are also making a package for students based on Python 3 and IPython.

At any rate, I would be VERY VERY interested to talk to you about the interaction amongst vagrant, python3, & ipython.

Running IPython Notebook over Python3 in a VM

2 Answers