Access data frames elements pyspark in jupyter

Question

I am new in spark and pyspark. My DataFrame is composed of several columns and Inside some columns, the is arrays or sub dataframes.

the printSchema of the df is in the image below

My question is how can I access the elements such as the "role" the "member" Thank you for your help

Answer 1

You want to access member or the other cols ? just do that :

df.select("group_profile.group.id", 
          "group_profile.group.members",
          "group_profile.intro",
          "group_profile.link",
          "group_profile.role",
         )