My lambda and EC2 are in a VPC and I cant putRecords to kinesis from lambda but I can from EC2

2
votes

I have a setup in aws, EC2 -> kinesis_x -> lambda -> kinesis_x (kinesis_x: to illustrate that this is that same kinesis stream) EC2 and lambda are in the same VPC. I can publish from lambda to kinesis_x.

I've configured a VPC endpoint, that did the trick but it disabled the EC2 from publishing to kinesis. Same security groups for EC2 and lambda.

After enabling the endpoint, this is the timeout exception from the java application in my EC2:

Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to kinesis.us-west-1.amazonaws.com:443 [kinesis.us-west-1.amazonaws.com/10.1.1.180, kinesis.us-west-1.amazonaws.com/10.1.2.157] failed: connect timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:150) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76)

I want to be able to publish to the same kinesis stream from EC2 and Lambda

2
amazon-web-servicesamazon-ec2amazon-kinesis

2 Answers

1
votes

Problem solved! We attached our security group to the Endpoint and that did the trick! This sg is the same as our EC2 and Lambda function has.

0
votes

Things to check, looking at the error the client cannot connect so:

  • Is the VPC public/private?
  • Are you using multiple subnets?
  • If private does it have an internet gateway?
  • What routes are setup?

All of the basic IP networking rules apply with a VPC.

Also telnet is your friend. A good trick is to SSH/SSM into the EC2 instance and try to connect using telnet eg

telnet ip port

This lets you isolate network connectivity from application-level issues.