How to implement non-blocking tcp server with ack mechanism?

Question

I am new to the multithreading web server programming Now I am writing a server program that:

1. Receive messages (in self-defined data format) from tcp socket
2. Process these messages (which takes time)
3. Send corresponding responses to the socket
4. Provide ACK mechanism for receiving messages and sending responses, that is every message contains a unique seq number and I should include the ack (same as seq) in the corresponding response. The other side also implements this mechanism. If I did not receive ACK from the other side for 5 min, I should re-send the message that I expected to receive corresponding ACK from.

My thought was to use a while loop to receive messages from the socket, then process the messages and send responses.

The problem is, processing messages takes time and I may receive multiple messages in a short period. So if I call the process_message() function in this while loop and wait for its finish, it will be blocking and I will definitely waste time. So I need non-blocking way.

I have done some research. I supposed I may use two common techs: thread pool and message queue.

For thread pool, my idea goes like the following pseudo code:

def process_message():
  process_message // takes time
  send_response(socket)

while True:
  message = recv(socket)
  thread = thread_pool.get_one()
  thread.start(target=process_message)

For message queue, I am not sure, but my idea would be having producer thread and consumer thread:

def consumer:
  // only one consumer thread?
  message = queue.poll()
  consumer_thread.process_message(message)
  send_response(socket)


while True:
  // only one producer thread?
  message = recv(socket)
  producer_thread.put_message_to_queue()

Hope my idea is clear. Can anyone provide some typical solution?

Then, the tricker part, any thoughts on how to implement the ACK mechanism?

Thank you!

Answer 1

This is rather broad because there is still too much to implement.

The general idea is indeed to implement:

• a TCP server, that will receive incoming messages and write them (including the socket from which they were received) in a queue
• a pool of worker threads that will get a message from the queue, process the message, and pass the response to an object in charge of sending the message and wait for the acknowledgement
• an object that will send the responses, store the sequence number, the socket and the message until the response has been acknowledged. A thread would be handy to process the list of message waiting for acknowledgement and sent them again when the timeout is exhausted.

But each part requires a consequent amount of work, and can be implemented in different ways (select, TCPServer or threads processing accepted sockets for the first, which data structure to store the messages waiting for acknowledgement for the third, and which pool implementation for the second). I have done some tests and realized that a complete answer would be far beyond what is expected on this site. IMHO, you'd better break the question in smaller answerable pieces, keeping this one as the general context.

You should also say whether the incoming messages should be immediately acknowledged when received or will be implicitely acknowledged by the response.