Create cluster with Shared Network in GKE

Question

I’m trying to create a cluster in GKE project-1 with shared network of project-2.

Roles given to Service account:
project-1: Kubernetes Engine Cluster Admin, Compute Network Admin, Kubernetes Engine Host Service Agent User
project-2: Kubernetes Engine Service Agent, Compute Network User, Kubernetes Engine Host Service Agent User

Service Account is created under project-1. API & Services are enabled in both Projects.

But I am getting this error persistently. Error: googleapi: Error 403: Kubernetes Engine Service Agent is missing required permissions on this project. See Troubleshooting | Kubernetes Engine Documentation | Google Cloud for more info: required “container.hostServiceAgent.use” permission(s) for “projects/project-2”., forbidden

data "google_compute_network" "shared_vpc" {
    name = "network-name-in-project-2"
    project = "project-2"
}

 
data "google_compute_subnetwork" "shared_subnet" {
    name = "subnet-name-in-project-2"
    project = "project-2"
    region = "us-east1"
}

 # cluster creation under project 1
 # project 1 specified in Provider 
resource "google_container_cluster" "mowx_cluster" {
    name = var.cluster_name
    location = "us-east1"
    initial_node_count = 1
 
    master_auth {
        username = ""
        password = ""
 
        client_certificate_config {
            issue_client_certificate = false
        }
    }
 
    remove_default_node_pool = true
    cluster_autoscaling {
        enabled = false
    }
 
    # cluster_ipv4_cidr = var.cluster_pod_cidr
    ip_allocation_policy {
        cluster_secondary_range_name = "pods"
        services_secondary_range_name = "svc"
    }
 
    network = data.google_compute_network.shared_vpc.id
    subnetwork = data.google_compute_subnetwork.shared_subnet.id
}

I think the key point you are missing: the Kubernetes Engine Service Agent needs the permission. This is a special type of service account assigned to Google Cloud services. — John Hanley
I added the service account (with name Google APIs Service Agent) of project 1 under project 2 and gave these two roles. 1. Compute network User 2. Kubernetes Engine Host Service Agent User but I am still getting the error. — xyphan
where can i find or create "Kubernetes Engine Service Agent for Project 1"? There are only two auto generate service accounts under project 1 IAM: 1. Compute Engine default service account 2. Google APIs Service Agent — xyphan
Service Accounts are listed under IAM -> Service Accounts. To change a Service Agent's roles, sometimes you must manually add the Service Agent email address to IAM. The documentation discusses this in detail. — John Hanley
So, only when you create a gke cluster via gcloud command, it auto creates "Kubernetes Engine Service Agent for Project 1". And then you can use terraform to create cluster. — xyphan

Wytrzymały Wiktor Wytrzymały Wiktor · Accepted Answer · 2021-01-20T09:55:25

This is a community wiki answer based on the discussion in the comments and posted for better visibility. Feel free to expand it.

The error you encountered:

Error: googleapi: Error 403: Kubernetes Engine Service Agent is missing required permissions on this project. See Troubleshooting | Kubernetes Engine Documentation | Google Cloud for more info: required “container.hostServiceAgent.use” permission(s) for “projects/project-2”., forbidden

means that the necessary service agent was not created:

roles/container.serviceAgent - Kubernetes Engine Service Agent:

Gives Kubernetes Engine account access to manage cluster resources. Includes access to service accounts.

The official troubleshooting docs describe a solution for such problems:

To resolve the issue, if you have removed the Kubernetes Engine Service Agent role from your Google Kubernetes Engine service account, add it back. Otherwise, you must re-enable the Kubernetes Engine API, which will correctly restore your service accounts and permissions. You can do this in the gcloud tool or the Cloud Console.

The solution above works as in your use case the account was missing so it had to be (re)created.

Create cluster with Shared Network in GKE

1 Answers