Good UDP relay server design

Question

I'm creating a UDP server that needs to receive UDP packets from various clients, and then forward them to other clients. I'm using C# so each UDP socket is a UdpClient for me. I think I want 2 UdpClient objects, one for receiving and one for sending. The receiving socket will be bound to a known port, and the sender will not be bound at all.

The server will get each packet, lookup the username in the packet data, and then based on a routing list the server maintains, it will forward the packet to 1 or more other clients.

I start the listening UdpClient with:

   UdpClient udpListener = new UdpClient(new IPEndPoint(ListenerIP, UdpListenerPort));
   udpListener.BeginReceive(new AsyncCallback(OnUDPRead), udpListener);

My callback looks like:

    void OnUDPRead(IAsyncResult ar)
    {
        UdpClient udpListener = (UdpClient)ar.AsyncState;

        try
        {
            IPEndPoint remoteEndPoint = null;
            byte[] packet = udpListener.EndReceive(ar, ref remoteEndPoint);

            // Get connection based on data in packet

            // Get connections routing table (from memory)

            // ***Send to each client in routing table***
        }
        catch (Exception ex)
        {
        }

        udpListener.BeginReceive(new AsyncCallback(OnUDPRead), udpListener);
    }

The server will need to process 100s of packets per second (this is for VOIP). The main part I'm not sure about, is what to do at the "Send to each client" part above. Should I use UdpClient.Send or UdpClient.BeginSend? Since this is for a real-time protocol, it wouldn't make sense to have several sends pending/buffered for any given client.

Since I only have one BeginReceive() posted at any given time, I assume my OnUDPRead function will never be called while already executing. Unless UdpClient works differently than TcpClient in this area? Since I don't give it a single buffer (I don't give it any buffer), I suppose it could fire OnUDPRead several times with only one BeginReceive posted?

Answer 1

I don't see any reason why you can't use one socket, the straightforward way. Async I/O doesn't appear to me to offer any startling benefit in this situation. Just use non-blocking mode. Receive a packet, send it. If send() causes EAGAIN or EWOULDBLOCK or whatever the C# manifestation of that is, just drop it.

In your plan, the sending socket will be bound automatically the first time you call send(). Clients will reply to that port. So it needs to be a port you are reading on. And you already have one of those.

Answer 2

It very much depends on scale - i.e. how many concurrent RTP streams you want to relay. I have developed exactly this solution as part of a media gateway for our SIP switch. Despite initial reservations, I have it working very efficiently using C# (1000 concurrent streams on a single server, 50K datagrams per second using 20ms PTime).

Through trial-and-error, I determined the following:

1) Using Socket.ReceiveFromAsync is much better than using synchronous approaches, since you don't need individual threads for each stream. As packets are received, they are scheduled through the threadpool.

2) Creating/disposing the SocketAsyncEventArgs structures takes time - better to allocate a pool of structures at startup and 'borrow' them from the pool.

3) Similarly for any other objects required during processing (e.g. a RTPPacket class). Allocating these dynamically at such a high rate results in GC issues pretty quickly with any sort of real load. If you avoid all dynamic allocation, and instead use pools of objects you manage yourself, this issue goes away.

4) If you need to do any clocking to co-ordinate input and output streams, transcoding, mixing or file playback, don't try to use the standard .NET timers. I wrapped the Win32 Multimedia timers from winmm.dll - these were much more precise.

I ended up with an architecture where components could expose IRTPSource and IRTPSink interfaces, which could then be hooked-up to create the graph required. I created RTP input/output components, mixer, player, recorder, (simple) transcoder, playout buffer etc. - and then wired them up using these two interfaces. The RTP packets are then clocked through the graph using the MM timers.

C# proved to be a good choice for what would typically be approached using C or C++.

Mike

Answer 3

What EJP said.

Most all the literature on scaling socket servers with async IO is written with TCP scenarios in mind. I did a lot of research last year on how to scale a UDP socket server, and found that IOCP/epoll wasn't as appropriate for UDP as it is for TCP. See this answer here.

So it's actually quite simple to scale a UDP socket server just using multiple threads than trying to do any sort of asynchronous I/O thing. And often times, one single thread pumping out recvfrom/sendto calls is good enough.

I would suggest that the server have one synchronous socket per client. Then have several threads do a Socket.Select call on a different subset of client sockets. I suppose you could have a single socket for all clients, but then you'll get some out-of-ordering problems if you try to process audio packets on that socket from multiple threads.

Also consider using the lower level socket class intead of UdpClient.

But if you really want to scale out - you may want to do the core work in C/C++. With C#, more layers of abstraction on top of winsock means more buffer copies and higher latency. There are techniques by which you can use winsock to forward packets from one socket to another without incurring a buffer copy.

Answer 4

a good design would be not to re-invent the wheel :)

There are already lots of RTP media proxy solutions: Kamailio, FreeSWITCH, Asterisk, and some other open-source projects. I have great experience with FreeSWITCH, and would highly recommend it as a media proxy for SIP calls.

If your signalling protocol is not SIP, there are some solutions for other protocols as well.