Socket UDP Python Server Looses Packets

Question

I'm using Python and I have a problem with UDP Socket as a server. I send packets from a client but the data is too large to send in one packet (several MBytes) so I fragment the data in packets of 65K (IP fragments it in 1500 bytes packets (MTU) and reassambles them in the server). I see the packets in Wiresharks so there isn't a problem in the transmission.

The transimision code is :

while(end == False):

        if total_length - count >= MTU:
            length_send = self._socket.sendto(msgbuff[count:count + MTU],(IP,PORT))
        else:
            length_send = self._socket.sendto(msgbuff[count:],(IP,PORT))

        count += length_send
        if count == total_length:
            end = True

The problem is in the recepction. I use a sync header to know where the datas begins (some fix bytes as 0xB451B451 for example). In some runs the server works well and find the sync header and it's able to recover all the packet.

However, in others runs the sockets only receives the last packets of the block data. For example : The block Data is 100K , the first packets is 65K and the second 35K. The sockets always returns the 35K packet.

The code of the server is:

while not synchronized_flag:
        msg, __ = self._socket.recvfrom(MTU)
        seq_sync = msg[0:length(SYNQ_HEADER)]
        if seq_sync == SYNQ_HEADER:
            synchronized_flag = True

total_length = getLength(msg)
bytes_read = 0
while bytes_read < total_length:
            msg_aux, _ = self._socket.recvfrom(MTU)
            msg.extend(msg_aux)
            bytes_read += len(msg_aux)

Thanks for your support.

In addition I must say that I can't use TCP because I need the highest speed I'm able to get.

TCP may in fact provide you with the highest speed you are able to get, once you’ve taken into account the handling of issues like packet loss and congestion control. Keep in mind that TCP was designed for reliable data transfer, and its designers faced all the same problems you are now discovering. If you want to stick with UDP you will likely find yourself creating your own TCP-like protocol over time; except your home-grown version won’t be as well-tested or reliable. — Jeremy Friesner

Gerd Gerd · Accepted Answer · 2020-03-11T14:26:20

Remember that UDP is not a reliable protocol and therefore often used for applications such as video streaming, but not if each and every packet needs to be received correctly.

You will have to take the decision if you want to establish a reliable connection (at the cost of more overhead and less speed) or a connection with minimal overhead (at the cost of potentially losing packets).

Socket UDP Python Server Looses Packets

1 Answers