Connecting to Google Cloud SQL using proxy -- Error 403: Insufficient Permission

Question

EDIT:

I now think the issue is with my Golang pod communicating with the proxy pod via localhost, as in the second error message.

I added the service account credentials JSON file to my Docker image's GOOGLE_APPLICATION_CREDENTIALS environment variable. After doing that, using my-project:us-central1:my-instance as connName below works.

However, when I try using the DB_HOST environment variable in the container as connName, I still get the 404 error below.

ORIGINAL POST

I'm following this guide to connect to Google Cloud SQL from a pod on Kubernetes Engine. The pod is running two containers: one with the Cloud SQL proxy image and another with a Golang service to do the actual database queries.

I'm getting the following error when my Golang service tries to initiate a connection:

ensure that the Cloud SQL API is enabled for your project (https://console.cloud.google.com/flows/enableapi?apiid=sqladmin). Error during createEphemeral for my-project:us-central1:my-instance: googleapi: Error 403: Insufficient Permission, insufficientPermission

I've looked at a few threads here and elsewhere and here's what I've done so far:

Ensured the Cloud SQL API is in fact enabled.
Added the Editor role to the service account I'm using.
Removed and re-added the Cloud SQL Client role on the service account I'm using.
Verified the correct secrets were created, with the same namespace as the pods.

Below is a snippet of the Golang code I'm using, which was taken from here:

cfg := mysql.Cfg(connName, dbUser, dbPassword)
cfg.DBName = dbName
db, err := mysql.DialCfg(cfg)

if err != nil {
    log.Println(err)
    return c.NoContent(http.StatusInternalServerError)
}

connName is the same string that shows up in the error: my-project:us-central1:my-instance. I've tried changing that to 127.0.0.1:3306 instead, but I then get this error below:

ensure that the account has access to "127.0.0.1:3306" (and make sure there's no typo in that name). Error during createEphemeral for 127.0.0.1:3306: googleapi: got HTTP response code 404 with body: Not Found

Also, here is a snippet of the yaml file I'm using to deploy the pods.

    env: 
    - name: DB_HOST
      value: 127.0.0.1:3306
    - name: DB_USER
      valueFrom:
        secretKeyRef:
          name: cloudsql-db-credentials
          key: username
    - name: DB_PASSWORD
      valueFrom:
        secretKeyRef:
          name: cloudsql-db-credentials
          key: password
  - name: cloudsql-proxy
    image: gcr.io/cloudsql-docker/gce-proxy:1.11
    command: ["/cloud_sql_proxy",
              "-instances=my-project:us-central1:my-instance=tcp:3306",
              "-credential_file=/secrets/cloudsql/credentials.json"]
    volumeMounts:
      - name: cloudsql-instance-credentials
        mountPath: /secrets/cloudsql
        readOnly: true
  volumes:
    - name: cloudsql-instance-credentials
      secret:
        secretName: cloudsql-instance-credentials

I've also verified that the Cloud SQL proxy starts without issue:

2018/04/21 20:41:19 Listening on 127.0.0.1:3306 for my-project:us-central1:my-instance
2018/04/21 20:41:19 Ready for new connections

I'm not sure what else to try here. Any help is appreciated.

@rexposadas I didn't, but I suspect specifying the port on the cloudsql-proxy container might fix it: ports: - containerPort: 3306 protocol: TCP — clk

Pol Arroyo Pol Arroyo · Accepted Answer · 2018-04-24T10:40:03

Hi as you can read here [1]: "If your program is written in Go you can use the Cloud SQL Proxy as a library, avoiding the need to start the Proxy as a companion process.", So you are already using the SQL proxy as code and there is no need to use the SQL proxy as a POD.

If you still want to use the SQL proxy as a POD, you can use the GO Companion Process as you can read here[2] under: "Companion Process":

import (
        "github.com/go-sql-driver/mysql"
)

dsn := fmt.Sprintf("%s:%s@tcp(%s)/%s",
       dbUser,
       dbPassword,
       "127.0.0.1:3306",
       dbName)
db, err := sql.Open("mysql", dsn)

[1] https://github.com/GoogleCloudPlatform/cloudsql-proxy#to-use-inside-a-go-program [2] https://cloud.google.com/sql/docs/mysql/connect-external-app#languages

Connecting to Google Cloud SQL using proxy -- Error 403: Insufficient Permission

1 Answers