Designing a C++ asio streaming server

Question

I'm trying to build a network server in C++ using the asio library(the non-boost version). I need to take images from a camera and stream them across the network. I toyed with asio before but this is the first time I'm trying to do something "meaningful" so I might have understood some things the wrong way. I'm not sure how to best handle in the context of asio sending the frames through the network.

I have a thread communicating with the camera backend getting frames as fast as the backend could produce them (e.g. 60 FPS, 120 FPS).

The asio configuration is:

• 1 io_service object
• one thread per CPU core to handle io_service::run()

Scenario 1:

The frames are copied, by a camera thread, in a triple buffer which is owned by a "server" entity. The server is notified by the camera thread each time a new frame is produced and it switches the buffers in order to get the "newest frame". The server listens and accepts multiple client connections async_accept.

For each connection the server sends the new frame buffer. Each connection issues an asio async_write. The server waits for all the completion handlers to finish before switching to a new frame as the buffer cannot be changed while asio write operations are in progress. In this scenarios there is no re-triggering of asio operations in the completion handlers, the asio operatios are issued only by the server.

Problems I see with this approach:

• As the server waits for the transmission to all the connected clients it might miss frames from the camera backend
• I have this as my current implementation and it does not scale well - the throughput drops quite drastically as I keep adding more clients
• Seems to be a poor usage of asio capabilities

Scenario 2:

The frames are copied, by a camera thread, in a buffer which is owned by a "server" entity. Each connection object holds its own triple buffer. The server is notified by the camera thread each time a new frame is produced and it copies the buffer to all the triple buffers that belong to the connections. That locks the server buffer for the duration of the copy. The server listens and accepts multiple client connections async_accept.

At the start of the streaming each connection switches its triple buffer in order to get the "newest frame" and issues an asio async_write. In this scenario each connection re-triggers the asio async_write operation in the completion handler.

Problems I see with this approach:

• As there is one triple buffer per connection the memory usage will increase significantly
• As the server copies the buffer to all the clients, it needs to lock it so it might miss frames from the source
• Faster clients might send the same frame twice if the server is not fast enough in updating the triple buffers

I would like to know if these approaches make sense in the context of asio, and what could be done differently to improve the design. I'm not looking for a definitive solution but more for a push in the right direction.

I looked extensively through the asio sample code but I could not find any example similar to mine.

Answer 1

First thing i would say, this is too big topic for a single SO thread. But some thoughts after reading ideas:

1. What about the network bandwidth?

With correct buffering ASIO can manage I/O rate, but if you want to serve many clients, bandwidth can be nearest bootleneck. Check out a calculator: even with 30fps with medium quality you can get to 28Mbps. In this case 1Gb network allow you to serve only 36 clients. Probably compression can help, but this is CPU tradeoff.

2. Memory consumption.

If camera continues to produce frames at lets say 60FPS, you should be able to deliver this data to clients with this speed. Probably it will not produce a problem, if a client is another server with good hardware and network link. But what if client has bad or laggy connection? In this case bandwidth will drop and your server should defer/slowdown writes to network. With infinite write buffers this will eat all of the RAM. Now, more problems if you have many clients and they have different receive speed. Handling this requires carefully checked buffering/write queue/frame drop mechanisms.

3. Requirements

I think there is some gap on requirement you have. Is 60FPS required minimum? What latency? What clients do you have? What if you cant send frames at this speed, how much frames you can drop in order to save memory? What hardware do they have? Do you have hardware limitations? I think its very important to answer this questions and many more and do the calculations, so you can get better answer how this can be done.