Cannot connect to RDS in another VPC via VPC peering

Question

I have two VPCs:

VPC A
- RDS instance
VPC B
- EC2 instance

There are also few subnets:

VPC A
- Private A
- Private B
- Peer A
VPC B
- Private A
- Private B
- Peer A

The RDS is in Private A, Private B, Peer A of VPC A.

The EC2 is in Peer A of VPC B.

I want to connect to the RDS instance from the EC2.

I have created a peering:

resource "aws_vpc_peering_connection" "a_to_b" {
  vpc_id      = aws_vpc.a.id
  peer_vpc_id = aws_vpc.b.id
  auto_accept = true

  accepter {
    allow_remote_vpc_dns_resolution = true
  }

  requester {
    allow_remote_vpc_dns_resolution = true
  }
}

resource "aws_vpc_peering_connection_accepter" "a_to_b" {
  vpc_peering_connection_id = aws_vpc_peering_connection.a_to_b.id
  auto_accept               = true
}

I also have route tables for the whole CIDR block like so:

resource "aws_route_table" "a_peer" {
  vpc_id = aws_vpc.a.id
}

resource "aws_route_table_association" "a_peer" {
  route_table_id = aws_route_table.a_peer.id
  subnet_id      = aws_subnet.a_peer.id
}

resource "aws_route" "a_peer_b" {
  route_table_id            = aws_route_table.a_peer.id
  destination_cidr_block    = aws_subnet.b_peer.cidr_block
  vpc_peering_connection_id = aws_vpc_peering_connection.a_to_b.id
}

resource "aws_route_table" "b_peer" {
  vpc_id = aws_vpc.b.id
}

resource "aws_route_table_association" "b_peer" {
  route_table_id = aws_route_table.b_peer.id
  subnet_id      = aws_subnet.b_peer.id
}

resource "aws_route" "b_peer_a" {
  route_table_id            = aws_route_table.b_peer.id
  destination_cidr_block    = aws_subnet.a_peer.cidr_block
  vpc_peering_connection_id = aws_vpc_peering_connection.a_to_b.id
}

I have also created security groups from ingress and egress on the RDS instance to the EC2 security group.

When I SSH into the EC2 I can get the DNS:

$ nslookup rds.xxxxxxxxxxx.eu-west-2.rds.amazonaws.com
Server:     192.16.0.2
Address:    192.16.0.2#53

Non-authoritative answer:
Name:   rds.xxxxxxxxxxx.eu-west-2.rds.amazonaws.com
Address: 10.16.192.135

However, curl cannot connect:

$ curl rds.xxxxxxxxxxx.eu-west-2.rds.amazonaws.com:5432

The expected response is:

$ curl rds.xxxxxxxxxxx.eu-west-2.rds.amazonaws.com:5432
curl: (52) Empty reply from server

The VPC peering is "Active" and the route tables match the Terraform.

How can I get this to connect?

Can you clarify your setup? Why do you route only to a single subnet cider range? Where is your second subnet for RDS? RDS must be in two AZs, thus it must use two subnets. How do you know that your RDS master is actually in this single subnet you specified, not the other one that is required? — Marcin
@Marcin The RDS is in multiple subnets: two private ones and the peered subnet. — sdgfsdh
You are setting up routes to only a single subnet, e.g. destination_cidr_block = aws_subnet.a_peer.cidr_block. What about the other subnets? Why not peer to VPC cidr as usually it is done. — Marcin
I do not want all of the resources in the other VPC to be accessible. Perhaps I am taking the wrong approach here? — sdgfsdh
Can you just try with VPC cider ranges in your route tables, rather then single subnets? — Marcin

Marcin Marcin · Accepted Answer · 2021-01-31T04:08:21

I did some tests on my own, and I'm pretty sure that the issue is caused by your routes, assuming that everything else in your VPC is correct as the VPCs and subnets definitions are not shown .

Specifically, you wrote that "RDS is in Private A, Private B, Peer A of VPC A". This means that RDS master may be in any of these subnets. You have no control over it, as its up to RDS to choose which subnet to use. You can only partially control it by selecting AZs when you create your RDS. Subsequently, your peering route tables should cover all these three subnets. The easiest way to achieve this is by using VPC CIDR range:

# Route from instance in VPC B to any subnet in VPC A which
# hosts your RDS in all its subnets
resource "aws_route" "b_peer_a" {
  route_table_id            = aws_route_table.b_peer.id
  destination_cidr_block    = aws_vpc.a.cidr_block
  vpc_peering_connection_id = aws_vpc_peering_connection.a_to_b.id
}

Then you also need to have a route table in VPC A associated with your peering connections for all its subnets:

resource "aws_route_table" "a_peer" {
  vpc_id = aws_vpc.a.id
}

resource "aws_route_table_association" "a_peer" {
  route_table_id = aws_route_table.a_peer.id
  subnet_id      = aws_subnet.a_peer.id
}

resource "aws_route_table_association" "a_private1" {
  route_table_id = aws_route_table.a_peer.id
  subnet_id      = aws_subnet.a_private1.id
}

resource "aws_route_table_association" "a_private2" {
  route_table_id = aws_route_table.a_peer.id
  subnet_id      = aws_subnet.a_private2.id
}

resource "aws_route" "a_peer_b" {
  route_table_id            = aws_route_table.a_peer.id
  destination_cidr_block    = aws_subnet.b_peer.cidr_block
  vpc_peering_connection_id = aws_vpc_peering_connection.a_to_b.id
}

Cannot connect to RDS in another VPC via VPC peering

1 Answers