You have a number of requirements, each of which can be built in Apex, but using different features:
Administrators may assign users to groups.
Only Administrators may access administrative controls.
Users may view privileges.
Users may only see information for their group.
I would use Apex Authorisation Schemes to take care of #1, #2 and #3. For example, create an authorization scheme called "Administrator" which checks whether the current user is an administrator, then apply this scheme to any pages, regions, buttons, items, etc. that should only be accessible to administrators.
For #4, there are a few solutions I can think of:
Predicates - make sure each query in the application checks whether the data is viewable by the current user according to their group.
Views - encapsulate the security predicates in a view on each table, so that you don't have to repeat this code throughout the application.
Oracle VPD or Row Level Security (requires Enterprise licence) - this hides the security predicates behind the SQL level.
I've used all 3 options above for different projects; the 3rd one was for a fairly large application with quite complex authority-checking rules. RLS made this project much simpler and easier to build and verify; however RLS may be overkill in some cases.