Using 'gcloud services vpc-peerings connect' in Deployment Manager

Question

I'm setting up a deployment manager package in .jinja which does the following: - Creates VPC Network, Subnet and Private Range for GCP Services - Creates the peering between 'servicenetworking.googleapis.com' and my VPC network - Assigns the Cloud SQL database into the private range assigned to google services within my VPC

The second step is proving impossible with deployment manager as there is no action that can be called to do this. I have confirmed the fix manually at this stage is to call the following gcloud command and then set up the Cloud SQL database in the VPC:

gcloud services vpc-peerings connect --service=servicenetworking.googleapis.com --ranges=<my-range> --network=<my-network> --project=<my-project>

Using the following gcp-type in my .jinja isn't sufficient as it doesn't allow mapping to pre-existing gcp services but expects a source and destination network.

- name: {{ env['deployment' ]}}-gcp-private-vpc-peering
  action: gcp-types/compute-v1:compute.networks.addPeering
  metadata:
    runtimePolicy:
    - CREATE
  properties:
    network: $(ref.{{ env['deployment']}}-network.name)
    name: {{ env['deployment' ]}}-gcp-private-vpc-peering
    autoCreateRoutes: true
    peerNetwork: servicenetworking.googleapis.com
    dependsOn:
    - $(ref.{{ env['deployment']}}-network.selfLink)

Is there a way to call the gcloud command from the deployment manager, or an action I can call upon to achieve the service peering. I can confirm the Service API is indeed enabled on the project.

(Note that the destination VPC and project are variable, assigned by Google therefore I cannot enter this value into the above template)

Update 05/07/19 I believe I have found the API service call I need to do, but I'm very unsure of the syntax to actually make the call to create the service link from deployment manager:

https://cloud.google.com/service-infrastructure/docs/service-networking/reference/rest/v1beta/services.connections/create

Need a bit of direction - similar to below?

- name: {{ env['deployment' ]}}-gcp-private-vpc-peering
  action:  gcp-types/servicenetworking.googleapis.com:services.connections
  metadata:
    runtimePolicy:
    - CREATE
  properties:
    propertyA: valueA
    ...

John W John W · Accepted Answer · 2019-11-26T20:17:25

@u-phoria

You're correct - and it's something that they currently have in their pipeline.

I've raised a product improvement ticket with them for it, it can be seen here:

Private VPC Peering for Cloud SQL is not supported by deployment manager. This resulted in the need to do the VPC peering from an escalated priviledge VM instance in the relevant VPC as it is the most secure option (updated 9th July 2019) https://issuetracker.google.com/137033144

An example of the resource you need to do this is shown below:

{# Bootstrapped box to complete the VPC Peering Setup #}
- name: {{ env['deployment'] }}-peering-setup
  type: compute.v1.instance
  properties:

    {# Checking whether the creation of new resources are specified #}
    {% if properties['createNewResources'] %}
    zone: {{ properties["zone"] }}
    machineType: https://www.googleapis.com/compute/v1/projects/{{ env["project"] }}/zones/{{ properties["zone"] }}/machineTypes/f1-micro
    networkInterfaces:
    - network: $(ref.{{ env['deployment']}}-network.selfLink)
      subnetwork: $(ref.{{ env['deployment']}}-subnetwork.selfLink)
      accessConfigs:
      - name: External NAT
        type: ONE_TO_ONE_NAT
    {% else %}
    zone: {{ common.ZONES[0] }}
    machineType: https://www.googleapis.com/compute/v1/projects/{{ env["project"] }}/zones/{{ common.ZONES[0] }}/machineTypes/f1-micro
    networkInterfaces:
    - network: https://www.googleapis.com/compute/v1/projects/{{ env["project"] }}/global/networks/default
      subnetwork: https://www.googleapis.com/compute/v1/projects/{{ env["project"] }}/regions/{{ common.REGION }}/subnetworks/default
      accessConfigs:
      - name: External NAT
        type: ONE_TO_ONE_NAT
    {% endif %}

    disks:
    - deviceName: boot
      type: PERSISTENT
      boot: true
      autoDelete: true
      initializeParams:
        sourceImage: https://www.googleapis.com/compute/v1/projects/debian-cloud/global/images/family/debian-9
    metadata:
      items:
      - key: startup-script
        value: |
          {# Creating VPC Peering to Google Services for the Managed Postgres in the existing network or the newly created one #}
          {% if properties['createNewResources'] %}
          #!/bin/bash
          sudo su -
          echo "Checking relevant peering connections to google services exist in local VPC" >> checking-status.sh
          output=$(gcloud services vpc-peerings list --network={{ env['deployment'] }}-network | grep "servicenetworking.googleapis.com")
          if [[ -z $output ]]; then
          echo "Peering not found, creating peering to servicenetworking.googleapis.com from private VPC" && gcloud services vpc-peerings connect --service=servicenetworking.googleapis.com --ranges={{ env['deployment'] }}-google-managed-services --network={{ env['deployment'] }}-network
          else
          echo "No peering created as relevant peering already exists"
          fi
          echo "Sending the signal to deployment manager to carry on with the deployment"
          gcloud beta runtime-config configs variables set success/{{ env['deployment'] }}-vpc-peering-setup success --config-name {{ env['deployment'] }}-startup-config

          echo "Destroying this instance now that it has succesfully executed peering change"
          gcloud compute instances delete --quiet --delete-disks=all --zone=europe-west1-b {{ env['deployment'] }}-peering-setup
          {% else %}
          #!/bin/bash
          sudo su -
          echo "Checking relevant peering connections to google services exist in local VPC" >> checking-status.sh
          output=$(gcloud services vpc-peerings list --network={{ properties['network'] }} | grep "servicenetworking.googleapis.com")

          if [[ -z $output ]]; then
          echo "Known GCP bug when re-creating GCP peering deployment into the same network reserved range, using the workaround published here: https://issuetracker.google.com/issues/118849070 and here https://github.com/terraform-providers/terraform-provider-google/issues/3294 to make sure this has no effect on the deployment"
          gcloud beta services vpc-peerings update --service=servicenetworking.googleapis.com --ranges={{ env['deployment'] }}-google-managed-services --network={{ properties['network'] }} --project={{ env['project'] }} --force

          echo "Peering not found, creating peering to servicenetworking.googleapis.com from private VPC" && gcloud services vpc-peerings connect --service=servicenetworking.googleapis.com --ranges={{ env['deployment'] }}-google-managed-services --network={{ properties['network'] }}
          else
          echo "No peering created as relevant peering already exists"
          fi
          echo "Sending the signal to deployment manager to carry on with the deployment"
          gcloud beta runtime-config configs variables set success/{{ env['deployment'] }}-vpc-peering-setup success --config-name {{ env['deployment'] }}-startup-config

          echo "Destroying this instance now that it has succesfully executed peering change"
          gcloud compute instances delete --quiet --delete-disks=all --zone=europe-west1-b {{ env['deployment'] }}-peering-setup
          {% endif %}
    serviceAccounts:
        - email: default
          scopes:
          - 'https://www.googleapis.com/auth/cloud-platform'
          - 'https://www.googleapis.com/auth/cloudruntimeconfig'
    dependsOn:
    - $(ref.{{ env['deployment'] }}-google-managed-services.selfLink)
    {% if properties['createNewResources'] %}
    - $(ref.{{ env['deployment'] }}-subnetwork.selfLink)
    {% endif %}

Whereby if the relevant parameters are set (in this case the createNewResources flag) it will create vpc peering between the two networks.

Keep in mind that you will also have to have a global address range setup for this before you execute the above jinja. An example of this is shown below:

- name: {{ env['deployment'] }}-google-managed-services
  type: compute.v1.globalAddresses
  properties:
    name: google-managed-services-{{ env['deployment'] }}
    {% if properties['createNewResources'] %}
    address: 10.73.144.0
    prefixLength: 20
    {% else %}
    address: {{ CIDRSplit[0] }}
    prefixLength: {{ CIDRSplit[1] }}
    {% endif %}
    addressType: INTERNAL
    purpose: VPC_PEERING

    {# Create the peering to the new network or the specified one #}
    {% if properties['createNewResources'] %}
    network: $(ref.{{ env['deployment']}}-network.selfLink)
    {% else %}
    network: https://www.googleapis.com/compute/v1/projects/{{ env["project"] }}/global/networks/{{ properties['network'] }}
    {% endif %}

    description: >
      Address range reserved for Google Managed Services.
      https://cloud.google.com/vpc/docs/configure-private-services-access

    {% if properties['createNewResources'] %}
    dependsOn:
    - $(ref.{{ env['deployment']}}-network.selfLink)
    {% endif %}

I hope this helps someone.

Using 'gcloud services vpc-peerings connect' in Deployment Manager

2 Answers