AWS Elastic Beanstalk unable to access AWS MSK

Question

I have an AWS MSK cluster running inside a VPC with 3 subnets.

When I created my Elastic Beanstalk (Java) environment it asked for VPC and I configured the same VPC where my MSK cluster is running. I also selected all the three listed subsets in my Elastic Beanstalk Network configuration. I did not assigned a public IP as I don't require access from internet to Elastic Beanstalk instances.

I also assigned AWS MSK Full Access permissions to the IAM Instance Profile that I selected for my Elastic Beanstalk environment under Security configuration. Just for information completeness, I selected the AWSServiceRoleForElasticBeanstalk as a service role.

On a side note, when I configured my Lambda to access the MSK cluster, it asked me for VPC as well as Security Groups explicitly. But I don't see any such configuration options for Security Groups in case of Elastic Beanstalk. Am I overlooking something here? my Lambda is able to successfully access MSK cluster.

I don't understand why my Elastic Beanstalk instance is unable to access my AWS MSK cluster. Am I missing something?

Answer 1

With the help of AWS Support, I was able to resolve this issue.

First, you can configure Security Groups under 'Instances' configuration card.

But, it was a bit confusing for me because, the VPC and Subnets are under 'Networking' configuration card, which is stacked way after the 'Instances' configuration card. And the Security Groups listed under 'Instances' directly depends on the VPC and Subnets selected under 'Networking'. If you change your selection in 'Networking' then you should update/review your Security Groups selection under 'Instances' as well.

So, in my case, first I select my target VPC and related Subnets under 'Networking' and only then I was able to see my target Security Groups under 'Instances'.