How to install pip packages in conda without root privilege?

Question

This will be a risky question because there are many seemingly easy solutions to similar problems which have already been answered but there are specific circumstances that I have, which makes them work non-ideally.

Standard conda workflow without pip

The standard workflow of installing a conda environment with an enviroments file is

conda env create -f environment.yml

This works all fine even without root access if you have the appropriate rights to the folder where you install Miniconda.

Standard conda workflow with pip

An environment.yml can contain pip packages. This is realised by parsing the packages from the environment.yml into a requirements.txt file and then executing a pip subprocess with the update and the requirements flags (more detail here).

This works in most of the cases fine, conda's pip install the relevant packages. However, if you lack the required privileges this can get troublesome, i.e if you run a pip install package_name inside an environment:

File "/secret_path_to_conda/miniconda3/envs/cheese/lib/python3.6/site-packages/pip/_internal/utils/unpacking.py", line 105, in set_extracted_file_to_default_mode_plus_executable
os.chmod(path, (0o777 & ~current_umask() | 0o111))

because a 777 access is required to the path, which is not available in my case to the right folder.

Temporary solutions

What works for me is a two step procedure:

1. Remove the pip files manually from the environment.yml and put them into a requirements.txt
2. Execute pip install --user requirements.txt

This installs the pip binaries into a place (~/.local/bin), where I have the required access and this works. It could naturally follow that, why don't I just install my entire Miniconda in that place, meaning that I wouldn't have access problems. This solution have already occured to me, however, the problem is that I'm limited to 8GB's of space in that space. This is not enough for all my conda dependencies of multiple projects.

What do I want?

Ideally, some other command which could replace

conda env create -f environment.yml

by internally calling pip install --user instead of plain pip install.

Similar questions and difference from similar problems

• this - conda's behaviour changed since that, and it's not a problem with access rights
• this - refers to pip only and not conda

Answer 1

If I can understand your problem correctly. You are not allowed to install packages inside default user directory because lack of space in it.

So how about to change installation directory when you using pip?

I didn't use this parameters before, but pip install --help saying us

-t, --target <dir>          Install packages into <dir>. By default this will not replace existing files/folders in <dir>. Use --upgrade to replace
--root <dir>                Install everything relative to this alternate root directory.
--prefix <dir>              Installation prefix where lib, bin and other top-level folders are placed
--user                      Install to the Python user install directory for your platform. Typically ~/.local/, or %APPDATA%\Python on Windows. (See the
                              Python documentation for site.USER_BASE for full details.)

So here is many parameters to customize path to install. Maybe choosing your environment path manually when using pip will be solution?
Also you may try to change USER_BASE path