How to create kubernetes cluster in google cloud platform in python using google-cloud-container module

Question

I'm trying to create kubernetes cluster on google cloud platform through python (3.7) using google-cloud-container module.

Created kubernetes cluster through google cloud platform and was able to successfully retrieve details for that cluster using google-cloud container (python module).

I'm trying now to create kubernetes cluster through this module. I created a JSON file with required key values and passed it as parameter, but i'm getting errors. Would appreciate if provided a sample code for creating kubernetes cluster in google cloud platform. Thank you in advance.

from google.oauth2 import service_account
from google.cloud import container_v1



class GoogleCloudKubernetesClient(object):

    def __init__(self, file, project_id, project_name, zone, cluster_id):
        credentials = service_account.Credentials.from_service_account_file(
            filename=file)
        self.client = container_v1.ClusterManagerClient(credentials=credentials)
        self.project_id = project_id
        self.zone = zone

    def create_cluster(self, cluster):
        print(cluster)
        response = self.client.create_cluster(self.project_id, self.zone, cluster=cluster)
        print(f"response for cluster creation: {response}")


def main():
    cluster_data = {
            "name": "test_cluster",
            "masterAuth": {
                "username": "admin",
                "clientCertificateConfig": {
                    "issueClientCertificate": True
                }
            },
            "loggingService": "logging.googleapis.com",
            "monitoringService": "monitoring.googleapis.com",
            "network": "projects/abhinav-215/global/networks/default",
            "addonsConfig": {
                "httpLoadBalancing": {},
                "horizontalPodAutoscaling": {},
                "kubernetesDashboard": {
                    "disabled": True
                },
                "istioConfig": {
                    "disabled": True
                }
            },
            "subnetwork": "projects/abhinav-215/regions/us-west1/subnetworks/default",
            "nodePools": [
                {
                    "name": "test-pool",
                    "config": {
                        "machineType": "n1-standard-1",
                        "diskSizeGb": 100,
                        "oauthScopes": [
                            "https://www.googleapis.com/auth/cloud-platform"
                        ],
                        "imageType": "COS",
                        "labels": {
                            "App": "web"
                        },
                        "serviceAccount": "[email protected]",
                        "diskType": "pd-standard"
                    },
                    "initialNodeCount": 3,
                    "autoscaling": {},
                    "management": {
                        "autoUpgrade": True,
                        "autoRepair": True
                    },
                    "version": "1.11.8-gke.6"
                }
            ],
            "locations": [
                "us-west1-a",
                "us-west1-b",
                "us-west1-c"
            ],
            "resourceLabels": {
                "stage": "dev"
            },
            "networkPolicy": {},
            "ipAllocationPolicy": {},
            "masterAuthorizedNetworksConfig": {},
            "maintenancePolicy": {
                "window": {
                    "dailyMaintenanceWindow": {
                        "startTime": "02:00"
                    }
                }
            },
            "privateClusterConfig": {},
            "databaseEncryption": {
                "state": "DECRYPTED"
            },
            "initialClusterVersion": "1.11.8-gke.6",
            "location": "us-west1-a"
        }


    kube = GoogleCloudKubernetesClient(file='/opt/key.json', project_id='abhinav-215', zone='us-west1-a')

    kube.create_cluster(cluster_data)


if __name__ == '__main__':
    main()

Actual Output:
Traceback (most recent call last):
  File "/opt/matilda_linux/matilda_linux_logtest/matilda_discovery/matilda_discovery/test/google_auth.py", line 118, in <module>
    main()
  File "/opt/matilda_linux/matilda_linux_logtest/matilda_discovery/matilda_discovery/test/google_auth.py", line 113, in main
    kube.create_cluster(cluster_data)
  File "/opt/matilda_linux/matilda_linux_logtest/matilda_discovery/matilda_discovery/test/google_auth.py", line 31, in create_cluster
    response = self.client.create_cluster(self.project_id, self.zone, cluster=cluster)
  File "/opt/matilda_discovery/venv/lib/python3.6/site-packages/google/cloud/container_v1/gapic/cluster_manager_client.py", line 407, in create_cluster
    project_id=project_id, zone=zone, cluster=cluster, parent=parent
ValueError: Protocol message Cluster has no "masterAuth" field.

Here is a trick for you. Create the cluster using the CLI gcloud. Add the debug options: --log-http --verbosity=debug. Now you know the exact details for the cluster data portion. — John Hanley

Nimrod Fiat Nimrod Fiat · Accepted Answer · 2020-08-12T09:39:50

Kind of late answer, but I had the same problem and figured it out. Worth writing for future viewers.

You should not write the field names in the cluster_data as they appear in the REST API. Instead you should translate them to how they would look by python convention, with words separated by underline instead of camelcase. Thus, instead of writing masterAuth, you should write master_auth. You should make similar changes to the rest of your fields, and then the script should work.

P.S you aren't using the project_name and cluster_id params in GoogleCloudKubernetesClient.init. Not sure what they are, but you should probably remove them.

How to create kubernetes cluster in google cloud platform in python using google-cloud-container module

2 Answers