Can we trigger AWS Lambda function from aws Glue PySpark job?

Question

Currently i'm able to run Glue PySpark job, but is this possible to call a lambda function from Glue this job ? Using below code from my PySpark Glue job i'm calling lambda function.

lambda_client = boto3.client('lambda', region_name='us-west-2')
response = lambda_client.invoke(FunctionName='test-lambda')

Error:

botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the Invoke operation: User: arn:aws:sts::208244724522:assumed-role/AWSGlueServiceRoleDefault/GlueJobRunnerSession is not authorized to perform: lambda:InvokeFunction on resource: arn:aws:lambda:us-west-2:208244724522:function:hw-test

But I added proper lambda roles to my Glue iam role, still getting above error. Any specific role need to add ?

Thanks.

I saw few links where this is not possible. Thats why raised a question here. — RK.
Below two roles attached to my Gluedefault role id : AWSLambdaBasicExecutionRole AWSLambdaVPCAccessExecutionRole — RK.

statut statut · Accepted Answer · 2018-11-14T11:42:14

To invoke AWS Lambda you can use the following policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowToExampleFunction",
            "Effect": "Allow",
            "Action": "lambda:InvokeFunction",
            "Resource": "arn:aws:lambda:<region>:<123456789012>:function:<example_function>"
        }
    ]
}

Your roles are not suitable for Lambda invocations as

AWSLambdaBasicExecutionRole – Grants permissions only for the Amazon CloudWatch Logs actions to write logs. You can use this policy if your Lambda function does not access any other AWS resources except writing logs.

AWSLambdaVPCAccessExecutionRole – Grants permissions for Amazon Elastic Compute Cloud (Amazon EC2) actions to manage elastic network interfaces (ENIs).

Please see documentation here about these roles.

Can we trigger AWS Lambda function from aws Glue PySpark job?

1 Answers