Group.ReadWrite.All is the graph API permission which is required for a service principal to allow creating groups in Azure AD. However, it would allow SPN to modify/delete any groups in the directory.
Could you please help to know how can we restrict the scope so the Service principal should only modify groups which it has created at the first place.