How do I create an AWS Lambda layer for seaborn?

Question

I have been unsuccessful in creating a layer for seaborn to be used on AWS Lambda.

I have tried creating a python environment and installing seaborn using the following commands:

python -m venv python
cd python/Scripts
activate
pip3 install seaborn

The package installs, and then I organize the folder using this heirarchy: aws_lambda_python37_layer_seaborn\python\lib\python3.7\site-packages\seaborn

(site-packages contains a number of other folders as well, which are dependencies of seaborn I assume)

I copied the heirarchy from a layer someone else created for matplotlib, which works correctly as a layer on AWS Lambda.

When I try to import seaborn however, I receive this error: "errorMessage": "Unable to import module 'lambda_function': No module named 'seaborn'"

I am using python 3.7

seaborn requires a lot of dependencies. I created the layer but it was over 70MB. Are you using two layers to overcome the limit of 50 MB per layer? — Marcin
I already have a working mathplotlib layer, which is a dependency of seaborn. Is that what you meant? — Joe Carter
I can provide the steps for the seaborn layer, the way I would do it, but I would have to exclude its dependencies due to the total size. If this would be helpful, I could provide an answer with details. — Marcin

Marcin Marcin · Accepted Answer · 2020-09-02T03:28:04

I just created the custom layer with seaborn and can confirm that it works. The layer explicitly installs some dependencies, but not uses numpy and scipy. These two packages must be provided by AWS managed layer AWSLambda-Python37-SciPy1x.

Thus you need to use two layers in your function. One custom created below, and the second one is AWSLambda-Python37-SciPy1x.

The technique used includes docker tool described in the recent AWS blog:

How do I create a Lambda layer using a simulated Lambda environment with Docker?

Thus for this question, I verified it as follows:

Create empty folder, e.g. mylayer.

mkdir mylayer && cd mylayer

Go to the folder and create requirements.txt:

echo seaborn > requirements.txt 
echo matplotlib >> requirements.txt
echo pyparsing >> requirements.txt
echo cycler >> requirements.txt
echo certifi >> requirements.txt
echo pillow >> requirements.txt
echo six >> requirements.txt
echo pandas >> requirements.txt
echo kiwisolver >> requirements.txt
echo pytz >> requirements.txt

Run the following docker command:

Note the use of --no-deps to skip any dependencies appart from those specified above.

docker run -v "$PWD":/var/task "lambci/lambda:build-python3.7" /bin/sh -c "pip install --no-deps -r requirements.txt -t python/lib/python3.7/site-packages/; exit"

Create layer as zip:

zip -9 -r mylayer.zip python

Create lambda layer based on mylayer.zip in the AWS Console. Don't forget to specify Compatible runtimes to python3.7.
Test the layer in lambda using the following lambda function:

import json

import seaborn

def lambda_handler(event, context):

    print(dir(seaborn))

The function executes correctly:

['FacetGrid', 'JointGrid', 'PairGrid', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', '_orig_rc_params', 'algorithms', 'axes_style', 'axisgrid', 'barplot', 'blend_palette', 'boxenplot', 'boxplot', 'categorical', 'catplot', 'choose_colorbrewer_palette', 'choose_cubehelix_palette', 'choose_dark_palette', 'choose_diverging_palette', 'choose_light_palette', 'clustermap', 'cm', 'color_palette', 'colors', 'countplot', 'crayon_palette', 'crayons', 'cubehelix_palette', 'dark_palette', 'desaturate', 'despine', 'distplot', 'distributions', 'diverging_palette', 'dogplot', 'external', 'factorplot', 'get_data_home', 'get_dataset_names', 'heatmap', 'hls_palette', 'husl_palette', 'jointplot', 'kdeplot', 'light_palette', 'lineplot', 'lmplot', 'load_dataset', 'lvplot', 'matrix', 'miscplot', 'mpl', 'mpl_palette', 'pairplot', 'palettes', 'palplot', 'plotting_context', 'pointplot', 'rcmod', 'regplot', 'regression', 'relational', 'relplot', 'reset_defaults', 'reset_orig', 'residplot', 'rugplot', 'saturate', 'scatterplot', 'set', 'set_color_codes', 'set_context', 'set_hls_values', 'set_palette', 'set_style', 'stripplot', 'swarmplot', 'utils', 'violinplot', 'widgets', 'xkcd_palette', 'xkcd_rgb']

How do I create an AWS Lambda layer for seaborn?

2 Answers