SignalR on Azure App Service with ARR Disabled

2
votes

Our server is scaling out with 1-3 instances in some specific period of time everyday. We have Azure Redis Backplane for connection persistency of signalr. In addition to this, the server doesn't ARR Affinity enabled. By the way we are using ServerSentEvents for Androids and WebSocket for iOS.

The problem is our mobile users(moto couriers) are disconnecting or reconnecting to SignalR server frequently because of their provider when the mobile signal is low.

We have checked all the things over mobile side. We pretty sure that we have one and only one signalr connection at a time. In addition to this when they are connected, we are storing their connectionids on persistent storage (SQL Database).

While sending a message to users we choose the latest connection id stored on Database. This means that we only sent to client's only one connection id.

However we get some feedbacks about the messages we sent over server popping up twice on their phones (Most of the time messages are received twice in the rush hours where server has 2 or 3 instances).

We are not able to trace it down why it is received twice especially on rush hours.

The question is, is there any chance this is about ARR Affinity? Because Redis backplane uses subscribe and publish methodology and since couriers are disconnecting/reconnecting frequently they have a chance to connect different servers, thus, when server sends a message, 2 servers might try to send that message and it is popping up on their phone twice even though they have one connection.

Additional info;

SignalR DisconnectTimeOut = 60 seconds

SignalR KeepAlive = 20 seconds

1
androidazureredissignalr
The only real way to fix things like this is to introduce ids and acks in your messages. That's how you guarantee that there's no duplicates (by having clients check the ids). As for why it's happening, it's hard to know. I don't think affinity would affect things for SignalR 2.x as messages still go through the backplane – davidfowl
Well, since the messages are coming ~ at the same time, the calling event is triggered for both messages. I need to have a lock/unlock for each message. Why I'm doubt about ARR is that we will hit everytime to the same instance so that SignalR will make a connection to the same instance everytime it disconnects/reconnects – kkocabiyik

1 Answers

1
votes

This seems to be the reason, when new connection request is made, you might need to drop existing one from other server, using replication.

if replication interval is short enough, it will minimise number of duplicates, for the rest, you might need to solve it on client side by ignoring last notification if notification hash/id is already received.