Server Sent Event (SSE) cluster connection handling

Question

At work we're currently building a webapp that supports SSE.
We're all relatively new to SSE, so we're dealing with lot's of (newbie) questions:

Disconnected connections

When a client closes his browsers we end up with a disconnected connection. Some call it ghost connections I think. How can we detect such connections at the server side? We want to delete those clients from the notification list.

Limit amount of connections

I understood that each 'SSE-connection' is kept alive on the application server. When I shut down my development server, I directly notice on the browser debugger that connection felt away. Shouldn't we set some upper limit of connections? The application server connections will get exhausted some time...
Additionally, some server dedicate a thread for each request. So that could lead to some thread exhausting issue...

Application or web server

Should the whole SSE-broadcasting be managed by an application server (where most request are business related) or should it be managed by some web server that is fully dedicated for handling SSE-event? At the moment, all business request and SSE-events are handled by a Jboss application server.

Clustered environment

How is SSE managed in a clustered environment applying an active-active mode (=master-master) where requests are routed randomly between instances?

If you have more useful information (and caveats to look out for) please feel free to share!

Answer 1

Clustered environment

By that I think you mean

• arbitrary number of backends
• a query from a client can target any backend instance (load balanced)
• backend instance can disappear silently (crash or pod is terminated for resource reasons or whatever ...)

In that case, I think you need to distinguish what needs to be pulled from the client and what needs to be pushed from the backend. Typically what needs to be pushed are real time notifications or updates. But many other calls can be pulled in a classical way. For everything that is pulled from client, as long as your backend is stateless then you are fine.

For everything that is pushed here is how I would do it : When opening the browser app (client) it creates a SSE connection to one of the backend instances. You have no guarantee of which one it is. The backend should send an heartbeat every X seconds to the client. If the client stop receiving the heartbeats then he can assume that the backend is down. It can then re-create the SSE connection. But you might have lost notifications... To avoid this you can maybe have a timestamp or a notification id, and when re-creating the connection you pass that timestamp or id to get all the subsequent notifications from it.

Underneath your backends, you need some sort of subscription system to a message bus or broker. So that each backend receives the notification and tries to push them to the connected clients.

Limit amount of connections

With the design above you can limit the number of connections per backend by simply killing the first one that was created. The client will then retry to establish the connection to potentially a new backend instance. This is not maybe not "clean" but may work...

Disconnected connections

See Audrey answer, this is the classical way of using heartbeat in fact :)

Application or web server

I think this question leads to opinionated answers. To me I would say "KISS" (keep it simple stupid)

Answer 2

Clustered environment

Managing a Server-Sent Event application is not very different from managing a traditional web application. You can use a load balancer to properly manage your clustered environment and implement an auto-scale mechanism to continue serving connections when you reach your instance limit.

The most important point you should be careful at is being sure you're using a proxy that can keep your connection open almost indefinitely: HAProxy and NGINX are good candidates.

Limit amount of connections

Some server recycle threads (Undertow for example) and some don't. Once you know how the server you chose is dealing with threads, you can calculate your limit per instance.

Application or web server

Except if your application server which manages "classic" business related requests is already heavily used, there is no need to have a dedicated server to manage SSE-events.

Disconnected connections

Concerning disconnected connections, you can test them by regularly sending what is usually called an "heartbeat" : an empty message which will return an error if the connexion has been closed by the customer.