I am working on a Lambda function that needs to import SciPy and pyarrow. Since there is an official AWS layer that contains SciPy - AWSLambda-Python38-SciPy1x:14 it is used in the function. pyarrow is contained in awswrangler that was already created as a layer to use in one of the previously made Lambdas.
AWS SAM is used to package and deploy the function. Now there is a pretty interesting difference between copying the contents of the awswrangler layer to the deployment package directory which can be deployed and executed correctly and when both SciPy and awswrangler layers are used in the function. If both layers are used, executing sam deploy gives the following error:
Layers consume more than the available size of 262144000 bytes (Service: AWSLambdaInternal; Status Code: 400; ...
One thing to note is that when awswrangler layer contents are copied to the deployment package, there is no overlap with the existing contents, so the deployment package is not smaller because f.e. it already contains some of the packages that awswrangler uses.
