Building TensorFlow package for AWS Lambda in python

3
votes

https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html#python-package-dependencies

Above link helps to build a deployment package for AWS-Lambda. I am planning to build a package to access TensorFlow. After following the steps from AWS documentation zip file is around 110MB and directory size is 330MB.

By using the below commands package size can be reduced:

find -name "*.so" | xargs strip

find -name "*.so*" |xargs strip

find . -name \*pyc -delete

rm -R wheel*

find . -type d -name "tests" -exec rm -rf {} +

Zip package size will be reduced to 90MB and directory size is around 290MB which is still greater than the threshold mentioned by AWS.

I also tried to use the docker to build AWS-Lambda package but, Again size was more than the AWS limit. https://medium.com/i-like-big-data-and-i-cannot-lie/how-to-create-an-aws-lambda-python-3-6-deployment-package-using-docker-d0e847207dd6

Does anyone have any suggestions/opinions where to find the correct documentation to build the package within the AWS size limit?

1
pythonpython-3.xtensorflowaws-lambda

1 Answers

0
votes

Don't build large dependencies into your lambda function. Use lambda layers to carry heavy dependencies. Lots of examples for this now. eg., https://github.com/antonpaquin/Tensorflow-Lambda-Layer