Google Cloud - Unable to access private CloudSQL database in one VPC network from another VPC network

0
votes

Our setup includes - A primary VPC where the we have compute engines and Postgres databases that are created with private IP. Let's refer it as main-network, - Vault deployed in it's own VPC and accessible via Loadbalancer (as per the best practice). Let's refer it as vault-network.

Within main-network compute instances are able to access the DBs with the private IP as the DBs where created with main-network as the parent network while creation. Looking at various VPC details, it seems like the creation process auto creates private-service-access as described in the docs..

The problem - For Vault database secret-engine, Vault needs to be able to access the DB to dynamically generate secrets. I have tried creating VPC network peering between main-network and vault-network and verified (via netcat) that I can successfully reach machines in the main-network from machines in the vault-network.

However, I cannot reach the DB instances from the nodes in vault-network.

Is it possible to share the access to private-service with a peered vpc network?

I don't want to make the DBs public unless it's the only way.

2
google-cloud-platformgoogle-cloud-sqlvpchashicorp-vault
VPC Peering is not transitive. This means you can connect from VPC A to VPC B. You cannot connect from VPC A thru VPC B to VPC C where Cloud SQL is located. You will need to peer Cloud SQL to VPC A. - John Hanley
Peering vault-network to CloudSql network was the first thing I tried. Unfortunately, it doesn't work as there is no option to select that particular cloudSql network anywhere. Would appreciate if you can add the steps - Raj Saxena
I explained what the problem is so that you would understand why. Review Guillaume's answer. - John Hanley

2 Answers

1
votes

Because your Cloud SQL is not in your VPC, but a VPC peering is performed between the Cloud SQL and your VPC, you can't reach another VPC network by peering. As John said, it's named transitivity and described in the VPC peering restrictions

Only directly peered networks can communicate. Transitive peering is not supported. In other words, if VPC network N1 is peered with N2 and N3, but N2 and N3 are not directly connected, VPC network N2 cannot communicate with VPC network N3 over VPC Network Peering.

Sadly, you can't create a peering between your Cloud SQL and the VPC network (your vault-network) of another project as suggested by John. I see only 2 solutions to solve this

  • set up a pass trough VM that only forward the traffic. Micro VM is enough for this, but you have to maintain it, to take care of the high availability, (...)
  • Set up a shared VPC whith your 2 projects inside. Set up your Cloud SQL private IP into the share VPC subnet.
0
votes

I reached out to the Google support for our account and asked them about this problem and they confirmed that it's not possible to have a transitive VPC peering

VPC peering is not transitive, this scenario is not possible using VPC peering between the VPC networks:

Cloud SQL <[VPC Peering]> VPC “main-network” <[VPC Peering]> VPC “vault-network”.

Creating a shared VPC defeats the purpose of creating a separate network in the first place as it would make the vault nodes directly accessible in the shared VPC.

The Google guy suggested to create a VPN:

As a workaround, you could create a VPN between the VPC networks “main-network” and “vault-network”:

Cloud SQL <[VPC Peering]> VPC “main-network” <[Cloud VPN]> VPC “vault-network”.

I need some time to think if we want to go down this path or not.

For now, I have whitelisted the NAT IP of the vault cluster in the settings.ip_configuration.authorized_networks configuration of the database instance to give access to the vault network. The down side of this is that the database has a public IP now which isn't necessarily bad as the firewall blocks public access to it.

Thanks to everyone for their suggestions.