AWS Lambda times out connecting to RedShift

Question

My Redshift cluster is in a private VPC. I've written the following AWS Lamba in Node.js which should connect to Redshift (dressed down for this question):

'use strict';
console.log('Loading function');

const pg = require('pg');

exports.handler = (event, context, callback) => {
var client = new pg.Client({
    user: 'myuser',
    database: 'mydatabase',
    password: 'mypassword',
    port: 5439,
    host: 'myhost.eu-west-1.redshift.amazonaws.com'
});


    // connect to our database
    console.log('Connecting...');
    client.connect(function (err) {
        if (err) throw err;

        console.log('CONNECTED!!!');

    });

};

I keep getting Task timed out after 60.00 seconds unfortunately. I see in the logs "Connecting...", but never "CONNECTED!!!".

Steps I've taken so far to get this to work:

As per Connect Lambda to Redshift in Different Availability Zones I have the Redshift cluster and the Lamba function in the same VPC
Also Redshift cluster and the Lamba function are on the same subnet
The Redshift cluster and the Lamba function share the same security group
Added an inbound rule at the security group of the Redshift cluster as per the suggestion here (https://github.com/awslabs/aws-lambda-redshift-loader/issues/86)
The IAM role associated with the Lamba Function has the following policies: AmazonDMSRedshiftS3Role, AmazonRedshiftFullAccess, AWSLambdaBasicExecutionRole, AWSLambdaVPCAccessExecutionRole, AWSLambdaENIManagementAccess scrambled together from this source: http://docs.aws.amazon.com/lambda/latest/dg/vpc.html (I realize I have some overlap here, but figured that it shouldn't matter)
Added Elastic IP to the Inbound rules of the Security Group as per an answer from a question listed prior (even if I don't even have a NAT gateway configured in the subnet)
I don't have Enhanced VPC Routing enabled because I figured that I don't need it.
Even tried it by adding the Inbound rule 0.0.0.0/0 ALL types, ALL protocols, ALL ports in the Security Group (following this question: Accessing Redshift from Lambda - Avoiding the 0.0.0.0/0 Security Group). But same issue!

So, does anyone have any suggestions as to what I should check?

*I should add that I am not a network expert, so perhaps I've made a mistake somewhere.

As far as I can see, you did everything fine. I have very similar setup, with the difference being our redshift clusters have public ip, for reasons different than lambda handling, and was changed from private to public at later time. I also use python, but shouldn't really matter. If you have aws support in your plan, I would use it. If you don't or want to tinker your way out of this one, setup a small ec2 machine in the same vpc as your redshift and lambda and see if you can connect from there. If you can, problem is probly with your lambda, if you can't, problem is likely with your vpc/RS. — Adam Owczarczyk
Thanks for your comment, but I used the public hostname while I should use the private IP (see accepted answer) — Danny van der Kraan
Ah crap, sorry, this tab was sitting in my browser since yesterday, didn't see the answer:) — Adam Owczarczyk
Possible duplicate of Conecting AWS Lambda to Redshift - Times out after 60 seconds — Hitesh

user818510 user818510 · Accepted Answer · 2017-05-22T19:15:58

The timeout is probably because your lambda in VPC cannot access Internet in order to connect to your cluster(you seem to be using the public hostname to connect). Your connection options depend on your cluster configuration. Since both your lambda function and cluster are in the same VPC, you should use the private IP of your cluster to connect to it. In your case, I think simply using the private IP should solve your problem.

Depending on whether your cluster is publicly accessible, there are some points to keep in mind.

If your cluster is configured to NOT be publicly accessible, you can use the private IP to connect to the cluster from your lambda running in a VPC and it should work.
If you have a publicly accessible cluster in a VPC, and you want to connect to it by using the private IP address from within the VPC, make sure the following VPC parameters to true/yes:
- DNS resolution
- DNS hostnames

The steps to verify/change these settings are given here.

If you do not set these parameters to true, connections from within VPC will resolve to the EIP instead of the private IP and your lambda won't be able to connect without having Internet access(which will need a NAT gateway or a NAT instance).

Also, an important note from the documentation here.

If you have an existing publicly accessible cluster in a VPC, connections from within the VPC will continue to use the EIP to connect to the cluster even with those parameters set until you resize the cluster. Any new clusters will follow the new behavior of using the private IP address when connecting to the publicly accessible cluster from within the same VPC.

AWS Lambda times out connecting to RedShift

3 Answers