How to set multiple environment variables in AWS Sagemaker notebook instance?

3
votes

I am using the AWS Sagemaker notebook instances for some of my experiments. As I am also using the lifecycle configurations scripts that are executed during notebook startup and also want to set some environment variables.

For some reason, when I set multiple env variables in the lifecycle shell script, they are not set by the instance, i.e. when I execute echo $FOO the relevant variable is not printed.

However, when I set only one env variable it is working and I can use it in my notebook session.

My understanding is that I start the desired Kernel,

I have also tried to set the env variables inside the notebook by running export FOO=BAR but that also did not work. Following the example script provided by AWS, I made my changes to set the variables, however when I print $FOO, it doesn't seem to be displayed.

I have tried setting the envs before and after switching to ec2-user (before the commands are executed as root), still nothing helped.

#!/bin/bash


cd /home/ec2-user/anaconda3/envs/python3
mkdir -p ./etc/conda/activate.d
mkdir -p ./etc/conda/deactivate.d
touch ./etc/conda/activate.d/env_vars.sh
touch ./etc/conda/deactivate.d/env_vars.sh

echo export FOO=BAR >> ./etc/conda/activate.d/env_vars.sh
echo unset FOO >> ./etc/conda/deactivate.d/env_vars.sh

echo export FOO2=BAR2 >> ./etc/conda/activate.d/env_vars.sh
echo unset FOO2 >> ./etc/conda/deactivate.d/env_vars.sh

sudo -u ec2-user -i <<'EOF'

cd /home/ec2-user

# This will affect only the Jupyter kernel called "conda_python3".
source activate python3

pip install --upgrade pip

pip install scipy xgboost sklearn

# You can also perform "conda install" here as well.


source deactivate

EOF

I want to set multiple environment variables for this Sagemaker notebook upon start, what is the best way to do this?

2
amazon-web-servicesjupyter-notebookcondaamazon-sagemaker

2 Answers

0
votes

Ok, so I have found the answer here. It seems that the way I export the environment variables was missing quotes so they were not exported properly.

Doing it like this, it works: 

#!/bin/bash 
set -e 
touch /home/ec2-user/anaconda3/envs/python3/etc/conda/activate.d/env_vars.sh 
echo "export MY_KEY='my-key-name'" >> /home/ec2-user/anaconda3/envs/python3/etc/conda/activate.d/env_vars.sh
0
votes

In AWS SageMaker Notebook Instance Lifecircle Configurations, create a script as the example listed below to set up environment variables. One could also refer to the official script examples. https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html

#!/bin/bash
set -e
# OVERVIEW
# This script sets username and email address in Git config
# PARAMETERS
YOUR_USER_NAME="***"
YOUR_EMAIL_ADDRESS="***"
sudo -u ec2-user -i <<EOF
git config --global user.name "$YOUR_USER_NAME"
git config --global user.email "$YOUR_EMAIL_ADDRESS"
EOF
# OVERVIEW
# This script sets environment variables
touch /etc/profile.d/jupyter-env.sh
echo "export TWILIO_ACCOUNT_SID=***" >> /etc/profile.d/jupyter-env.sh
echo "export TWILIO_AUTH_TOKEN=***" >> /etc/profile.d/jupyter-env.sh
initctl restart jupyter-server --no-wait