Conda environment from .yaml offline

2
votes

I would like to create a Conda environment from a .yaml file on an offline machine (i.e. no Internet access). On an online machine this works perfectly fine:

conda env create -f environment.yaml

However, it doesn't work on an offline machine as the packages are then not found. How do I do this?

If that's not possible is there another easy way to get my complete Conda environment to an offline machine (including both Conda and pip installed packages)?

Going through the packages one by one to install them from the .tar.bz2 files works, but it is quite cumbersome, so I would like to avoid that.

2
yamlofflineconda
If these are packages that conda retrieves from PyPI? Any special reason not to use the standard pip for installing? - Anthon
Hi, Thanks for the response! I would be fine with installing them with pip but I also don't know how to do that offline without going through the packages one by one. I would be very thankful for any hints on that. - lilaf

2 Answers

1
votes

If you can use pip to install the packages, you should take a look at devpi, particutlarily its server. devpi can cache packages normally installed from PyPI, so only on first install it actually retrieves them. You have to configure pip to retrieve the packages from the devpi server.

As you don't want to list all the packages and their dependencies by hand you should, on a machine connected to the internet:

  • install the devpi server (I have that running in a Docker container)
  • run your installation
  • examine the devpi repository and gathered all the .tar.bz2 and .whl files out of there (you might be able to tar the whole thing)

On the non-connected machine:

  • Install the devpi server and client
  • use the devpi client to upload all the packages you gathered (using devpi upload) to the devpi server
  • make sure you have pip configured to look at the devpi server
  • run pip, it will find all the packages on the local server.

devpi has a small learning curve, which already worth traversing because of the speed up and the ability to install private packages (i.e. not uploaded to PyPI) as a normal dependency, by just generating the package and upload it to your local devpi server.

0
votes

I guess that Anthon's solution above is pretty good but just in case anybody is interested in an easy solution that worked for me:

I first created a .yaml file specifying the environment using conda env export > file.yaml. Following the instructions on http://support.esri.com/en/technical-article/000014951, I automatically downloaded all the necessary installation files for conda installed packages and created a channel from the files. For that, I just adapted the code from the link above to work with my .yaml file instead of the conda list file they used. In addition, I automatically downloaded the necessary files for the pip installed packages by looping through the pip entries in the .yaml file and using pip download for downloading each of them. Furthermore, I automatically created separate conda and pip requirement lists from the .yaml file. Then I created the environment using conda create with the offline flag, the file with the conda requirements and my custom channel. Finally, I installed the pip requirements using pip install with the pip requirements file and the folder containing the pip installation files for the option --find-links.

That worked for me. The only problem is that you can only download binaries with pip download if you need to specify a different operating system than the one you are running, and for some packages no binaries are available. That was okay for me now as the target machine has the some characteristics but might be problem in the future, so I am planning to look into the solution suggested by Anthon.