Scaling Socket.IO with Redis and then scaling Redis itself

4
votes

After using Node.js and socket.io for a while, I understand that if I want my application to support up to 1 million concurrent users I need to scale it, So I started using Redis to PUB/SUB messages between sockets and running a lot of socket.io servers instances, in the same machine and on other machines but all my socket.io servers works with the same Redis server.

This makes me think: what's the point? Will I need a few more redis servers and scale between them? My point is that their will always be a bottleneck on the top server.

My question is, is it possible to scale Redis? And if yes, how will all my sockets that connected to different socket.io server be able to PUB/SUB between the Redis servers?

2
javascriptnode.jsredissocket.ionode-redis
please, do you use the same connection for the pub/sub ? thanks – user1255808
No, creating 2 clients for each connected client – udidu
terribly sorry, I would like say for no the pub/sub. So ? thank you – user1255808
You read this post of mine @ stackoverflow.com/a/4446424/11926 ? That example can be written much better! Then one instance of redis can scale all your socket.io servers! – Alfred

2 Answers

2
votes

I think what you're looking for is Redis Cluster. It is mentioned on that page as "being worked on" and "hopefully for this summer".

There's a presentation here. It's basically hashing + single master and slaves for redundancy. But it's not ready yet. Your best bet for more information is to hit up the forums and IRC for those products.

If someone is doing something similar, they will be on those channels.

0
votes

For further readers, using socket.io with its redis adapter is not scalable (at least for now). Under the hood, redis adapter uses pub/sub to broadcast. Here's a discussion about that: https://github.com/redis/redis/issues/2672

1 thing to highlight: Redis Cluster is not the answer (discussed in the issue above or see youtube link below to get more detail)

I found one interesting solution on scaling Redis pubsub here: https://redislabs.com/wp-content/uploads/2018/04/Redis-Day-TLV-2018-Scaling-Redis-PubSub.pdf

Watch youtube: https://www.youtube.com/watch?v=6G22a5Iooqk&ab_channel=RedisLabs