0
votes

I have a backend process that does work on my database. That's used on a separate computer so that way the frontend works miracles (in terms of speed at least). That backend process creates a UDP server and listen for packets on it.

On the frontend computer, I create child process from a server. Each child may create data in the database that require the backend to do some more work. To let the backend know, I send a PING using a UDP client connection.

  Front End / Backend Setup                        Processing

+-------+         +---------+                     +----------+
|       |         |         |                     | Internet |
| Front |  PING   | Backend |                     | Client   |
|  End  |-------->|         |                     +----------+
|       |         |         |              HTTP Request | 
+-------+         +---------+                           v
    ^                  ^                          +----------+
    |                  |                          | FrontEnd |--------+
    |                  |                          +----------+   PING |
    v                  v                  HTTP Response |             v
+---------------------------+                           v         +---------+
|                           |                     +----------+    | Backend |
|    Cassandra  Database    |                     | Internet |    +---------+
|                           |                     | Client   |
+---------------------------+                     +----------+

Without the PING, the backends ends its work and falls asleep until the next PING wakes it up. Although there is a failsafe, I put a timeout of 5 minutes so the backend wakes up once in a while no matter what.

My question here is about the UDP stack, I understand it is a FIFO, but I am wondering about two parameters:

  1. How many PING can I receive before the FIFO gets full?

  2. May I receive a PING and lose it if I don't read it soon enough?

The answer to these questions can help me adjust the current waiting loop of the backend server. So far I have assumed that the FIFO had a limit and that I may lose some packets, but I have not implemented a way to allow for packets disappearing (i.e. someone sends a PING, but the backend takes too long before checking the UDP stack again and thus the network decides that the packet has now timed out and removes it from under my feet.)


Update: I added a simple processing to show what happens when (it is time based from top to bottom)

2
UDP packets are not guaranteed to be received. Why don't you program the server to send a return packet, so at least you can see that the ping was received? Failing that, send a ping often enough so that statistically it is extremely unlikely that the server will fall asleep.Robert Harvey
The frontend must run as fast as possible. It is answering an HTTP request so I do not want it to wait on anything. Since I would imagine that there will be enough frontends pinging, what you mention should be taken care of by the very nature of the system. What I'm wondering is whether packets may be withdrawn. On a rather slow system, it could otherwise take 5 minutes for the backend to start doing work that could have been done in seconds...Alexis Wilke
Your backend system should be acknowledging received packets immediately. If it could take 5 minutes for the server to respond to a packet, you should consider some form of asynchronous processing, like Node.JS uses. 5 minutes is a bit too long for synchronous requests.Robert Harvey
I expect the frontend to be gone by the time the backend is done processing the new data (the data in the database). The use of the backend is to do processing that the frontend can skip on so that way it goes a lot faster to send a reply to the client. The 5 min. is a safe guard just in case the backend misses all the pings, somehow... This will work in a local network environment so I don't expect losing so many UDP packets that the 5 min. would end up being the only mean of action.Alexis Wilke

2 Answers

1
votes

You can set the default buffer size on your system with sysctl, or set it per socket using setsockopt with the SO_RCVBUF option.

int n = 512 * 1024; // 512K
if (setsockopt(my_socket, SOL_SOCKET, SO_RCVBUF, &n, sizeof(n)) == -1) {
   perror("Failed to set buffer size, using default");
}

There is also a maximum set on the system that you can't go over. On my machine default receiving buffer size is 208K and max is 4M:

# sysctl net.core.rmem_max
net.core.rmem_max = 4194304
# sysctl net.core.rmem_default
net.core.rmem_default = 212992
1
votes

How many PING can I receive before the FIFO gets full?

It depends on the size of your socket receive buffer.

May I receive a PING and lose it if I don't read it soon enough?

Yes and no. Datagrams which have been received and which fit into the socket receive buffer remain there until they have been read or the socket is closed. You can tune the size of the socket receive buffer within limits. However a datagram that arrives when the socket receive buffer is full are dropped.