amazon web services - AWS Lambda import module error in python

Question

I am creating a AWS Lambda python deployment package. I am using one external dependency requests . I installed the external dependency using the AWS documentation http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html. Below is my python code.

import requests

print('Loading function')

s3 = boto3.client('s3')


def lambda_handler(event, context):
    #print("Received event: " + json.dumps(event, indent=2))

    # Get the object from the event and show its content type
    bucket = event['Records'][0]['s3']['bucket']['name']
    key = urllib.unquote_plus(event['Records'][0]['s3']['object']['key']).decode('utf8')
    try:
        response = s3.get_object(Bucket=bucket, Key=key)
        s3.download_file(bucket,key, '/tmp/data.txt')
        lines = [line.rstrip('\n') for line in open('/tmp/data.txt')]
        for line in lines:
            col=line.split(',')
            print(col[5],col[6])
        print("CONTENT TYPE: " + response['ContentType'])
        return response['ContentType']
    except Exception as e:
        print(e)
        print('Error getting object {} from bucket {}. Make sure they exist and your bucket is in the same region as this function.'.format(key, bucket))
        raise e

Created the Zip the content of the project-dir directory and uploaded to the lambda(Zip the directory content, not the directory). When I am execute the function I am getting the below mentioned error.

START RequestId: 9e64e2c7-d0c3-11e5-b34e-75c7fb49d058 Version: $LATEST
**Unable to import module 'lambda_function': No module named lambda_function**

END RequestId: 9e64e2c7-d0c3-11e5-b34e-75c7fb49d058
REPORT RequestId: 9e64e2c7-d0c3-11e5-b34e-75c7fb49d058  Duration: 19.63 ms  Billed Duration: 100 ms     Memory Size: 128 MB Max Memory Used: 9 MB

Kindly help me to debug the error.

Answer 1

Error was due to file name of the lambda function. While creating the lambda function it will ask for Lambda function handler. You have to name it as your Python_File_Name.Method_Name. In this scenario I named it as lambda.lambda_handler (lambda.py is the file name).

Please find below the snapshot.

Answer 2

If you are uploading a zip file. Make sure that you are zipping the contents of the directory and not the directory itself.

Answer 3

Another source of this problem is the permissions on the file that is zipped. It MUST be at least world-wide readable. (min chmod 444)

I ran the following on the python file before zipping it and it worked fine.

chmod u=rwx,go=r

Answer 4

I found Nithin's answer very helpful. Here is a specific walk-through:

Look-up these values:

1. The name of the lambda_handler function in your python script. The name used in the AWS examples is "lambda_handler" looking like "def lambda_handler(event, context)". In this case, the value is "lambda_handler"
2. In the Lambda dashboard, find the name of the Handler in the "Handler" text-box in the "Configuration" section in the lambda dashboard for the function (shown in Nithin's screenshot). My default name was "lambda_function.lambda_handler".
3. The name of your python script. Let's say it's "cool.py"

With these values, you would need to rename the handler (shown in the screenshot) to "cool.lambda_handler". This is one way to get rid of the "Unable to import module 'lambda_function'" errorMessage. If you were to rename the handler in your python script to "sup" then you'd need to rename the handler in the lambda dashboard to "cool.sup"

Answer 5

Here's a quick step through.

Assume you have a folder called deploy, with your lambda file inside call lambda_function.py. Let's assume this file looks something like this. (p1 and p2 represent third-party packages.)

import p1
import p2

def lambda_handler(event, context):
    # more code here

    return {
        "status": 200,
        "body" : "Hello from Lambda!",
    }

For every third-party dependency, you need to pip install <third-party-package> --target . from within the deploy folder.

pip install p1 --target .
pip install p2 --target .

Once you've done this, here's what your structure should look like.

deploy/
├── lambda_function.py
├── p1/
│   ├── __init__.py
│   ├── a.py
│   ├── b.py
│   └── c.py
└── p2/
    ├── __init__.py
    ├── x.py
    ├── y.py
    └── z.py

Finally, you need to zip all the contents within the deploy folder to a compressed file. On a Mac or Linux, the command would look like zip -r ../deploy.zip * from within the deploy folder. Note that the -r flag is for recursive subfolders.

The structure of the file zip file should mirror the original folder.

deploy.zip/
├── lambda_function.py
├── p1/
│   ├── __init__.py
│   ├── a.py
│   ├── b.py
│   └── c.py
└── p2/
    ├── __init__.py
    ├── x.py
    ├── y.py
    └── z.py

Upload the zip file and specify the <file_name>.<function_name> for Lambda to enter into your process, such as lambda_function.lambda_handler for the example above.

Answer 6

There are just so many gotchas when creating deployment packages for AWS Lambda (for Python). I have spent hours and hours on debugging sessions until I found a formula that rarely fails.

I have created a script that automates the entire process and therefore makes it less error prone. I have also wrote tutorial that explains how everything works. You may want to check it out:

Hassle-Free Python Lambda Deployment [Tutorial + Script]

Answer 7

I found this hard way after trying all of the solutions above. If you're using sub-directories in the zip file, ensure you include the __init__.py file in each of the sub-directories and that worked for me.

Answer 8

I had the error too. Turn out that my zip file include the code parent folder. When I unzip and inspect the zip file, the lambda_function file is under parent folder ./lambda.

Use the zip command, fix the error:

zip -r ../lambda.zip ./*

Answer 9

In lambda_handler the format must be lambda_filename.lambda_functionName. Supposing you want to run the lambda_handler function and it's in lambda_fuction.py, then your handler format is lambda_function.lambda_handler.

Another reason for getting this error is module dependencies.

Your lambda_fuction.py must be in the root directory of the zip file.

Answer 10

The issue here that the Python version used to build your Lambda function dependencies (on your own machine) is different than the selected Python version for your Lambda function. This case is common especially if the Numpy library part of your dependencies.

Example: Your machine's python version: 3.6 ---> Lambda python version 3.6

Answer 11

@nithin, AWS released layers concept inside Lambda functions. You can create your layer and there you can upload as much as libraries and then you can connect the layer with the lambda functions. For more details: https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html

Answer 12

A perspective from 2019:

AWS Lambda now supports Python 3.7, which many people (including myself) choose to use as the runtime for inline lambdas.

Then I had to import an external dependency and I followed AWS Docs as the OP referred to. (local install --> zip --> upload).

I had the import module error and I realised my local PC had Python 2.7 as the default Python. When I invoked pip, it installed my dependency for Python 2.7.

So I switched locally to the Python version that matches the selected runtime version in the lambda console and then re-installed the external dependencies. This solved the problem for me. E.g.:

$ python3 -m pip install --target path/to/lambda_file <external_dependency_name>

Answer 13

Make sure that you are zipping all dependencies in a folder structure as python/[Your All Dependencies] to make it work as per this documentation.

https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html#configuration-layers-path

Answer 14

I ran into the same issue, this was an exercise as part of a tutorial on lynda.com if I'm not wrong. The mistake I made was not selecting the runtime as Python 3.6 which is an option in the lamda function console.

Answer 15

Sharing my solution for the same issue, just in case it helps anyone.

Issue: I got error: "[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': No module named 'StringIO'" while executing aws-big-data-blog code[1] provided in AWS article[2].

Solution: Changed Runtime from Python 3.7 to Python 2.7

[1] — https://github.com/bsnively/aws-big-data-blog/blob/master/aws-blog-vpcflowlogs-athena-quicksight/CloudwatchLogsToFirehose/lambdacode.py [2] — https://aws.amazon.com/blogs/big-data/analyzing-vpc-flow-logs-with-amazon-kinesis-firehose-amazon-athena-and-amazon-quicksight/

Answer 16

You can configure your Lambda function to pull in additional code and content in the form of layers. A layer is a ZIP archive that contains libraries, a custom runtime, or other dependencies. With layers, you can use libraries in your function without needing to include them in your deployment package. Layers let you keep your deployment package small, which makes development easier.

References:-

1. https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html
2. https://towardsdatascience.com/introduction-to-amazon-lambda-layers-and-boto3-using-python3-39bd390add17

Answer 17

You need to zip all the requirements, use this script

#!/usr/bin/env bash
rm package.zip
mkdir package
pip install -r requirements.txt --target package
cat $1 > package/lambda_function.py
cd package
zip -r9 "../package.zip" .
cd ..
rm -rf package

use with:

package.sh <python_file>

Answer 18

My problem was that the .py file and dependencies were not in the zip's "root" directory. e.g the path of libraries and lambda function .py must be:

<lambda_function_name>.py
<name of library>/foo/bar/

not

/foo/bar/<name of library>/foo2/bar2

For example:

drwxr-xr-x  3.0 unx        0 bx stor 20-Apr-17 19:43 boto3/ec2/__pycache__/
-rw-r--r--  3.0 unx      192 bx defX 20-Apr-17 19:43 boto3/ec2/__pycache__/__init__.cpython-37.pyc
-rw-r--r--  3.0 unx      758 bx defX 20-Apr-17 19:43 boto3/ec2/__pycache__/deletetags.cpython-37.pyc
-rw-r--r--  3.0 unx      965 bx defX 20-Apr-17 19:43 boto3/ec2/__pycache__/createtags.cpython-37.pyc
-rw-r--r--  3.0 unx     7781 tx defN 20-Apr-17 20:33 download-cs-sensors-to-s3.py

Answer 19

Actually go to the main folder (deployment package)that you want to zip,

Inside that folder select all files and then create the zip and upload that zip

Answer 20

Please add below one after Import requests

import boto3

What I can see that is missing in your code.

Answer 21

Your package directories in your zip must be world readable too.

To identify if this is your problem (Linux) use:

find $ZIP_SOURCE -type d -not -perm /001 -printf %M\ "%p\n"

To fix use:

find $ZIP_SOURCE  -type d -not -perm /001 -exec chmod o+x {} \;

File readable is also a requirement. To identify if this is your problem use:

find $ZIP_SOURCE -type f -not -perm /004 -printf %M\ "%p\n"

To fix use:

find $ZIP_SOURCE  -type f -not -perm /004 -exec chmod o+r {} \;

If you had this problem and you are working in Linux, check that umask is appropriately set when creating or checking out of git your python packages e.g. put this in you packaging script or .bashrc:

umask 0002 

Answer 22

change python file name to lambda_function.py, for example in my case the filename was app.py the error gone once i change filename to lambda_function.py

Answer 23

TL;DR, try chmod 755 python

I know this is a very long thread already, but if you still can't solve the following issue after reading all the potential solutions posted here by all the community members, you can try the solution that solved my problem.

{
  "errorMessage": "Unable to import module 'xxx': No module named 'yyy'",
  "errorType": "Runtime.ImportModuleError",
  "stackTrace": []
}

To be clear, I'm talking about the zip file for the AWS Lambda Layer (NOT the deployment package) for the Python runtime. In my case, all the dependencies are placed under a folder named python according to the doc.

Note that using a folder named python is better than the other folder structure since it does not depend on the version of Python.

Then you should try doing an ls in Lambda using the following code snippet:

import os

def lambda_handler(event, context):
    print(os.listdir('/opt'))
    print(os.listdir('/opt/python'))

Here is the expected output:

['python']
['OpenSSL', '__pycache__', ... 'urllib3', 'urllib3-1.24.2.dist-info']

In my case, before the issue was solved, the output was

{
  "errorMessage": "[Errno 13] Permission denied: '/opt/python'",
  "errorType": "PermissionError",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 16, in lambda_handler\n    print(os.listdir('/opt/python'))\n"
  ]
}

Therefore, it's clearly a permission issue. After doing a chmod 755 python, zip -r9 lambda-layer python, and uploading lambda-layer.zip to Lambda, the issue is solved.

Answer 24

No need to do that mess.

use python-lambda

https://github.com/nficano/python-lambda

with single command pylambda deploy it will automatically deploy your function