1
votes

I have over one hundred GCP users that have mixed roles. These users were manually created via IAM console by different admin personnel. We would like do some clean up to the project. The roles needed for these users are simply project viewer, bigquery user, bigquery data viewer and bigquery jobuser. I would like to change the roles of these 100+ users with a single command. Is there a way to do this?

Below is an example of IAM list that I retrieved from GCP command

gcloud projects get-iam-policy example-project-id-1

This is simply a dummy data and a modified version for security reasons.

---
policy:
  bindings:
  - members:
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]

    role: roles/editor
project: projects/926557397521
resource: //cloudresourcemanager.googleapis.com/projects/abc-staging
---
policy:
  bindings:
  - members:
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]

    roles:roles/bigquery.dataEditor
project: projects/926557397521
resource: //cloudresourcemanager.googleapis.com/projects/abc-staging
---
policy:
  bindings:
  - members:
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]
    - user:[email protected]

    roles:roles/bigquery.jobUser
project: projects/926557397521
resource: //cloudresourcemanager.googleapis.com/projects/abc-staging
---
policy:
  bindings:
  - members:
    - user:[email protected]
    role: roles/editor
project: projects/926557397521
resource: //iam.googleapis.com/projects/abc-staging/serviceAccounts/[email protected]

So, what i would like to do from the above example is to remove the the project editor role and simply assign them roles of BigQuery Data Viewer and BigQuery Job User

2
For with single command you are looking exactly for a single command or an easy way to do this? Top of my head I think about Terraform, but not sure if you are looking for one command operation or an automatic way to do it.Armando Cuevas
What I meant by with with single command was an easy way to do it, not just one sentence of command. An automatic way to do itbusheriff
@busheriff Do you have a .csv file with a list of all the users you want to change the permissions already?Alexandre Moraes
You can build a script that do this, but it's not "easy" as a simple command (or few list of commands). And, if they have all the same roles, the best way is to use a Google Groupguillaume blaquiere
Hi, I have modified the questions so that I could show what I wanted to do a bit clearerbusheriff

2 Answers

1
votes

Since you mentioned in the comment section that this policy binding file contains only the users that you want to change the policies. I was able to create a small process which will help you change your user's policy bindings for your project.

Below it is detailed described with the necessary code to execute each step, in order to change your user's permissions:

1. Get your Policy Bindings file and save it in your shell environment in yaml format, as follows:

gcloud projects get-iam-policy PROJECT_ID --format yaml > policy.yaml

2. Make a copy of the current policy bindings for security reasons with the command

cp policy.yaml your_policy_bindings_file_SECURITY_COPY.yaml

3. Within your Cloud Shell environment create a python file with the below code, name it python_script.py. This script extracts the user's emails from your policy binding YAML file and create a csv file, with the format described here (Upload users from a CSV file). In addition, it creates another .csv file only with the emails. So, it can be used to remove the current policies for each of these users. Below is the code:

import yaml
import re
import pandas as pd
import csv

#declaring the list which will contain all the user's emails from the policy biding yaml
final_list=[]

def yaml_as_python(val):
    """Convert YAML to dict"""
    try:
        return yaml.load_all(val,Loader=yaml.FullLoader)
    except yaml.YAMLError as exc:
        return exc

with open('policy.yaml','r') as input_file, open('emails_to_remove_policies.csv', 'w') as emails_to_remove_policies:
    results = yaml.load_all(input_file,Loader=yaml.FullLoader)

    #print(results)
    for value in results:

        try:
            mydict=value['policy']['bindings'][0]['members']
            mydict=sorted(mydict)
            r=re.compile(r'.+\@(abc.com)')
            my_newlist=[email.lstrip("user:") for email in mydict if re.match(r,email)]
            
            final_list.extend(my_newlist)
            #final_list.writerow(email_list)

            
        except TypeError:
            #print("type erorr")
            continue
            
    #writing the csv with the list of emails which will be removed from the policies        
    wr = csv.writer(emails_to_remove_policies, quoting=csv.QUOTE_NONE, delimiter = '\n')
    wr.writerow(final_list)

#CSV file example from documentation
#Header: Group Email [Required],Member Email,Member Type,Member Role
#Entry: [email protected], [email protected],USER,MEMBER 


df = pd.DataFrame(final_list)
df2 = pd.DataFrame(final_list, columns=['Member Email'])
df2 = df2.assign(GroupEmail='[email protected]').assign(MemberType='USER').assign(MemberRole= 'MEMBER')
df2 = df2.reindex(columns=['GroupEmail','Member Email','MemberType','MemberRole'])


df2 = df2.rename(columns={'GroupEmail': 'Group Email [Required]', 'MemberType': 'Member Email', 
                          'MemberRole': 'Member Role' })
#in case you want to check the final output in the terminal
#print (df2)
print("Finished")

#list of emails to add in the google group
df2.to_csv('emails_to_add.csv',index=False)
print("\n\nStarting removing the policies for the list of users\n\n")

Notice that, two .csv files will be created: emails_to_remove_policies.csv and emails_to_add.csv. The first one, will be used within a bash script to remove the current policies for these users from your project. The second will be used to add the emails to a Google Group, then this group will be used to grant the appropriate permissions within your policy bindings project yaml file.

4. Create a file named exec_all.sh with the following code to execute the python_script.py and remove the current policies for the list of unwanted users.

python3 python_script.py

while IFS=, read -r email
do
    for i in $email
        do
        echo "Removing $i"
        gcloud projects remove-iam-policy-binding PROJECT_ID --member user:$i --role ROLE_ID
        #echo "Next"
        done
done < emails_2.csv

From your shell environment execute it by typing: bash exec_all.sh.

5. Now, create a Google Group following the steps described here. Afterwards, add multiple members to your Google Group, you have to provide a .csv with an specific format containing all the users, use the emails_to_add.csv output from step two. Here is the documentation.

In order to download the .csv file to your local environment and then upload it to your Google Group using Google Admin, use the following command:

cloudshell download emails_to_add.csv

6. Get your new Policy Bindings file, after removing the unwanted users in step 4. As follows:

gcloud projects get-iam-policy PROJECT_ID --format yaml > new_policy.yaml

7. Open it, you can use cat, vim or the shell editor, which is the easiest method. Now, in the beginning of the file add:

---
bindings:
- members:
  - user: [email protected]
  role: role_id
---

Attention to the --- on the first and last line. Also, in order to get the role id, you have to go to: IAM & Admin > Roles (on bottom of the left menu) > Click on the Role you want > You can see the role id next to ID. Finally, notice that the group email is used to grant permission to all the desired users.

8. You have your new Policy Bindings for your project. Thus, you need to set it as the new policy for your project, here is the documentation. You can do it as below:

gcloud projects set-iam-policy PROJECT_ID new_policy.yaml

Notes:

  • You have to prepare your environment in order to run the codes above, installing the necessary packages. Here is the documentation for it.
  • Notice that, from your IAM policies from your project, all the unwanted users are removed. Then, they are added back to the project with the group alias and the proper permissions. We do this because, there are other permissions in your project which I assume you do not want to change (also, that is also the reason, we do a security copy in the beginning).
0
votes

GCP recommend collecting users with the same responsibilities into groups and assigning Cloud IAM roles to the groups rather than to individual users.

This will solve your problem and you will get additional benefits like easy onboarding/off boarding the users, easy to add/remove permissions, etc..,

Reference:

https://cloud.google.com/docs/enterprise/best-practices-for-enterprise-organizations#groups-and-service-accounts