9
votes

I am trying to set up firehose to send data from a kinesis stream to a redshift cluster. Firehose successfully inserts the data to my s3 bucket, but I am receiving the following error when firehose attempts to execute the s3->Redshift copy command:

The connection to the specified Amazon Redshift cluster failed. Ensure that security settings allow Firehose connections, that the cluster or database specified in the Amazon Redshift destination configuration JDBC URL is correct, and that the cluster is available.

I have performed every setup step according to this except for one: I did not make my Redshift cluster publicly accessible. I am unable to do this bc the cluster is in a private VPC that does not have an internet gateway attached.

After researching the issue, I found this article which provides insight for how to set up an AWS PrivateLink with firehose. However, I have heard that some AWS services support PrivateLink and others do not. Would PrivateLink work for this case?

I am also concerned with how this would affect the security of my VPC. Could anyone provide insight to possible risks to using a PrivateLink?

1
Did you find a solution. I'm facing the same issue. - navig8tr
Unfortunately not. I ended up using a separate VPC and making the Redshift cluster publicly accessible. - ketcham

1 Answers

0
votes

I were able to solve this issue. You need to add Internet gateway to your VPC route table. Goto to Redshift VPC

On the Routes tab (you must have 3 private route), choose Edit, Add another route, and add the following routes as necessary. Choose Save when you're done.

For IPv4 traffic, specify 0.0.0.0/0 in the Destination box, and select the internet gateway ID in the Target list.

If you add internet gateway ID to all your 3 private route, you might see Failure in other applications those are using that same route/VPC . To fix that, update only 1 route with internet gateway ID and rest two will have nat as destination for (0.0.0.0/0).