Server sent events, Puma, Rails and max dedicated threads for each client

Question

I am using Redis for my rails project to subscribe to channels and publishing to those channels when an event occurs. On the client side, I am registering to EventSource that correspond to these channels. Whenever an event occurs for the subscribed channel at the server, the server does a SSE write so that all registered clients receive the update.

Now the connection with the server stays alive for each client that is subscribed to those channels i.e. server-thread dedicated to this client keeps running until the client disconnects. With this approach, if there are 1000 concurrent users subscribed to a channel, I'd have 1000 TCP/IP connections open.

I am using Puma as the web server as suggested in this tutorial. Puma by default specifies 16 max threads. I can change this limit to a higher limit.

I might not know how many concurrent users there might be at a time in my app and do not know what max no. of threads I can specify in Puma. In a worst case scenario, if the count of threads dedicated to each concurrent user reaches the max count of the threads specified for the Puma webserver, the app would freeze for all users until one of the concurrent user disconnects.

I was excited to use Rails live streaming, and server sent events in my rails project but with this approach I risk to reach the limit of max threads specified in my web server and consequently app getting unresponsive for all users till one of the concurrent user disconnects.

Not sure what is the typical max thread count for Puma for a large concurrent user-base.

Should I consider other approaches - perhaps ajax-based polling or Node.js that uses an event-driven, non-blocking I/O model? Or just run some benchmarks to know what my max thread count can be?

Answer 1

I am actually working on a project where we went with polling because of the open connection issues. We figured it would be easier to just poll every three seconds, then keep a connection open and hanging. But the data freshness requirement weren't very strict being three seconds, so it was doable, and kind of silly to waste a thread for three seconds.

So unless you have very strict requirements for data freshness, and/or have a restricted user base, and/or ability to have a lot of threads, regular polling is normally the way to go.

And on the other hand if they are going to be constantly hitting your server, and it would take longer to poll the data again then your data freshness requirement, you might as well keep the connection open to avoid having to deal with the whole stack again.

Also in puma 2, you can also run it in cluster mode, which means it spawns an addition workers with there own threads and you can end up with Workers X Threads = Total Threads. Which might help in your calculations.

https://github.com/puma/puma#clustered-mode