I'm looking for a work around to use numpy in AWS lambda. I am not using EC2 just lambda for this so if anyone has a suggestion that'd be appreciated. Currently getting the error:
cannot import name 'multiarray'
Using grunt lambda to create the zip file and upload the function code. All the modules that I use are installed into a folder called python_modules inside the root of the lambda function which includes numpy using pip install and a requirements.txt file.
Updated to include the solution here, rather than a link:
After much effort, I found that I had to create my deployment package from within a python3.6 virtualenv, rather than directly from the host machine. I did the following within a Ubuntu 16.04 docker image. This assumes that you have python3.6, virtualenv and awscli already installed/configured, and that your lambda function code is in the ~/lambda_code directory:
1) cd ~ (We'll build the virtualenv in the home directory)
2) virtualenv venv --python=python3.6 (Create the virtual environment)
3) source venv/bin/activate (Activate the virtual environment)
4) pip install numpy
5) cp -r ~/venv/lib/python3.6/site-packages/* ~/lambda_code (Copy all installed packages into root level of lambda_code directory. This will include a few unnecessary files, but you can remove those yourself if needed)
6) cd ~/lambda_code
7) zip -r9 ~/package.zip . (Zip up the lambda package)
8) aws lambda update-function-code --function-name my_lambda_function --zip-file fileb://~/package.zip (Upload to AWS)
Your lambda function should now be able to import numpy with no problems.
If you want a more out-of-the-box solution, you could consider using serverless to deploy your lambda function. Before I found the above solution, I followed the guide here and was able to run numpy successfully in a python3.6 lambda function.
As of 2018 it's best to just use the inbuilt layers functionality.
AWS have actually released a pre-made one with numpy in it: https://aws.amazon.com/blogs/aws/new-for-aws-lambda-use-any-programming-language-and-share-common-components/
I would add this answer as well: https://stackoverflow.com/a/52508839/1073691
Using pipenv includes all of the needed .so files as well.
I was unable to find a good solution using serverless plugins, but I did find a good way with layers. See Serverless - Numpy - Unable to find good bind path format
1.) Do a Pip install of numpy to a folder on your local machine.
2.) once complete, zip the entire folder and create a zip file.
3.) Go to AWS lambda console, create a layer and upload zip file created in step 2 there and save the layer.
4.) After you create your lambda function, click add layer and add the layer you created. That's it, import numpy will start working.
