Makefile can't use `conda activate`

Question

I need to activate a conda environment in my makefile in order to run some python scripts, however, whenever I try to run conda activate env_name, I get the following message:

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'. If your shell is Bash or a Bourne variant, enable conda for the current user with

$ echo ". /Users/MY_USERNAME/anaconda3/etc/profile.d/conda.sh" >> ~/.bash_profile

or, for all users, enable conda with

$ sudo ln -s /Users/MY_USERNAME/anaconda3/etc/profile.d/conda.sh /etc/profile.d/conda.sh

The options above will permanently enable the 'conda' command, but they do NOT put conda's base (root) environment on PATH. To do so, run

$ conda activate

in your terminal, or to put the base environment on PATH permanently, run

$ echo "conda activate" >> ~/.bash_profile

Previous to conda 4.4, the recommended way to activate conda was to modify PATH in your ~/.bash_profile file. You should manually remove the line that looks like

export PATH="/Users/MY_USERNAME/anaconda3/bin:$PATH"

^^^ The above line should NO LONGER be in your ~/.bash_profile file! ^^^

I've tried changing the shell for the makefile by adding SHELL := /bin/zsh at the top, but this doesn't fix the problem. Additionally, I need this makefile to be able to run using whatever the default shell is for the computer (some of my teammates use zsh, others use bash). It seems like no matter what I do, I can't get conda activate to work in the makefile.

What can I do to get it to work, or is this impossible?

Answer 1

After a bit of searching around, I came up with adding this pattern to my Makefile to make conda activate work. Others may be able to simplify.

# Need to specify bash in order for conda activate to work.
SHELL=/bin/bash
# Note that the extra activate is needed to ensure that the activate floats env to the front of PATH
CONDA_ACTIVATE=source $$(conda info --base)/etc/profile.d/conda.sh ; conda activate ; conda activate

py3build:
    ($(CONDA_ACTIVATE) py3.6 ; python setup.py build )

Answer 2

You should use .ONESHELL: directive at beginning of script. This run all in the same shell.

Answer 3

One important thing you need to remember is that Makefile will execute each line of the recipe in a separate sub-shell so e.g. exporting PATH in one line will not affect what the command in the next Makefile line can see! Please refer to the Makefile manual - Recipe Execution:

When it is time to execute recipes to update a target, they are executed by invoking a new sub-shell for each line of the recipe, unless the .ONESHELL special target is in effect (see Using One Shell) (In practice, make may take shortcuts that do not affect the results.)

Please note: this implies that setting shell variables and invoking shell commands such as cd that set a context local to each process will not affect the following lines in the recipe.

Answer 4

conda activate among other things set environment variables. However, GNU Make invokes each line of recipe in a newly spawned shell. You would need to invoke conda activate in each line of the recipe.

A better way is for makefile to do conda activate and then re-run itself in the newly activated environment and only then build your targets.