AWS Lambda invoke function doesn't always return

Question

I am running an AWS Lambda function for image/video processing with Node 4.3 as a runtime. I am invoking my function from my webapp code with the node aws-sdk.

The issue is that when the function takes a long time to execute (e.g. 250 seconds), the invocation callback is never received, although I can clearly see in the aws cloudwatch logs that the function executed properly. On top of that the function is re-run again at least twice (this is probably related to the maxRetries parameter from the invocation: since it doesn't get a response back it retries).

What confuses me is that it works for quicker functions and my lambda function never times out. Anyone having such an issue ? Can it be related to the new 4.3 runtime? Note that I omit the context.succeed()or context.fail() as it is not required anymore, but I tried with it and it doesn't change a thing.

Lambda code

...
var handler = function (event, context, callback) {
    // video/image processing code
    // 
    // callback function
    ..., function(err, result) {
        if (err) {
            console.log("Error", err, err.stack);
            callback(err);
        } else {
            console.log("Done");
            callback(null, result);
        }
    }
};

Lambda Invocation

var lambda = new AWS.Lambda;

var myEventObject = {...};
var payload = JSON.stringify('myEventObject');

var params = {
    FunctionName: 'myLambdaFunction'
    InvocationType: 'RequestResponse',
    LogType: 'None',
    Payload: payload
};

lambda.invoke(params, function(err, data) {
    if (err) console.log(err, err.stack); // an error occurred
    else     console.log(data);           // successful response
});

Lambda CloudWatch Log

REPORT RequestId: xxx   Duration: 206757.82 ms  Billed Duration: 206800 ms Memory Size: 1536 MB Max Memory Used: 354 MB

I suspect that the timeout config for the sdk is 240 seconds, which could explain that you get a retry. Could you try to change this config within you cli? — Tom
No the timeout config is set at max: 300 seconds. If it goes beyond 300 seconds the functions times out and I can see it in the cloudwatch logs. If I set the timeout to a lower value, say 30 seconds, the function returns with a timeout error and I catch the callback error. — Daniel T
please note that i m not talking about the timeout of lambda, but the one from the sdk config, so do you? — Tom
Oh sorry, my bad. I indeed missed this setting when looking at the aws doc. It works by adding the appropriate option which default at 120000 : var lambda = new AWS.Lambda({httpOptions{timeout: 300000}}); Thank you so much — Daniel T
happy to help :) putting this comment as an answer for potential future readers — Tom

Tom Tom · Accepted Answer · 2016-04-16T10:20:57

as stated within the comments, the aws sdk has its own timeout config setting, that is by default set to 120 seconds (this is a different timeout setting than the lambda one).

in the current case, as the lambda runs for more this timeout, the sdk is assuming a failure and firing retries.

setting this config to 300 seconds should solve the problem. As Daniel T showed in the comment, temporary changing this can be achieved this way: var lambda = new AWS.Lambda({httpOptions{timeout: 300000}});

AWS Lambda invoke function doesn't always return

1 Answers