Activate Anaconda Python environment from makefile

Question

I want to use a makefile to create my project environment using a makefile and anaconda / miniconda , so I should be able to clone the repo and just run make myproject

 myproject: build build: @printf "\nBuilding Python Environment\n" @conda env create --quiet --force --file environment.yml @source /home/vagrant/miniconda/bin/activate myproject 

If I try this, I get the following error

make: source: Command not found

make: *** [source] Error 127

I was looking for a solution, but [this question / answer ( How to install a script in a Makefile? ) Assumes that I cannot use source from inside the makefile.

This response , however, offers a solution to (and received several upvotes), but it does not work for me either

(\
source / home / vagrant / miniconda / bin / activate myproject; \

)

/ bin / sh: 2: source: not found

make: *** [source] Error 127

I also tried moving the source activate step to a separate bash script and executing this script from the makefile. This does not work, and I guess for the same reason, that is, I am running source from within the shell.

I should add that if I run source activate myproject from the terminal, it works correctly.

Answer 1

I had a similar problem; I wanted to create or update a conda environment from the Makefile to make sure my own scripts can use python from this conda environment.
By default, make uses sh to execute commands, and sh does not know the source (also see this SO answer ). I just installed SHELL in bash and ended up with (for the relevant part only):

 SHELL=/bin/bash CONDAROOT = /my/path/to/miniconda2 . . install: sometarget source $(CONDAROOT)/bin/activate && conda env create -p conda -f environment.yml && source deactivate 

Hope this helps

Answer 2

I had the same problem. Essentially, the only solution is 9000. I have a script installation shell inside which I configure the conda environment (the source activates python2), then I invoke the make command. I experimented with setting up the environment from the Makefile and without success.

I have this line in my makefile:

 installpy : ./setuppython2.sh && python setup.py install 

Error messages:

 make ./setuppython2.sh && python setup.py install running install error: can't create or remove files in install directory The following error occurred while trying to add or remove files in the installation directory: [Errno 13] Permission denied: '/usr/lib/python2.7/site-packages/test-easy-install-29183.write-test' 

Essentially, I was able to configure the conda environment to use my local conda, which I have access to. But this is not picked up by the make process. I do not understand why the environment installed in my shell script using the "source" is not visible in the make process; The original command should change the current shell. I just want to share this so that other people do not try to do this. I know autotoools has a way to work with python. But make is probably limited in this regard.

My current solution is a shell script:

cat py2make.sh

 #!/bin/sh # the prefix should be change to the target # of installation or pwd of the build system PREFIX=/some/path CONDA_HOME=$PREFIX/anaconda3 PATH=$CONDA_HOME/bin:$PATH unset PYTHONPATH export PREFIX CONDA_HOME PATH source activate python2 make 

This seems to work well for me.

In a similar situation, there was a solution, but it doesn't seem to work for me:

My modified segment Makefile:

 installpy : ( source activate python2; python setup.py install ) 

Error message after calling make:

 make ( source activate python2; python setup.py install ) /bin/sh: line 0: source: activate: file not found make: *** [installpy] Error 1 

I don’t know where I am mistaken. If anyone has a better solution, share it.

Answer 3

You must use this; it is functional for me at the moment.

 report.ipynb : merged.ipynb ( bash -c "source ${HOME}/anaconda3/bin/activate py27; which -a python; \ jupyter nbconvert \ --to notebook \ --ExecutePreprocessor.kernel_name=python2 \ --ExecutePreprocessor.timeout=3000 \ --execute merged.ipynb \ --output=$< $<" )