If what you are trying to do is create a mesh network with audio and video flowing at all times to everyone via a mesh network then I'd suggest reconsidering. Getting this to more than 3 or 4 users to work well is hard to impossible to achieve. See here for an explanation of what goes on the network for different network topologies: http://testrtc.com/different-multiparty-video-conferencing/
If you are trying to do this to get data connected (non voice or video), then you can do that, but again, the number of peer connections you can place in a single browser is limited and each one you add puts some overhead. Today, I wouldn't try to get this over 40 or 50 at most.
By your question, I am assuming all clients will be connected to your server via a WebSocket. You will probably end up routing messages through the server and creating peer connections judiciously when you want to connect a specific user to another one - or a small group of users together.
For the group scenario, I'd use an SFU model - look at Jitsi (https://jitsi.org/Projects/JitsiVideobridge) and Kurento (http://www.kurento.org/) for possible alternatives.