Python UDP socket misses packets

Question

I'm implementing client-server communication using UDP that's used for FTP. First off, you don't need to tell me that UDP is unreliable, I know. My approach is: client asks for a file, server blasts the client with udp packets with sequence numbers, then says "what'd you miss?", resending those. On a local network, packet loss is < 1%. I'm pretty new to socket programming, so I'm not familiar with all the socket options (of which most examples found on google are for tcp).

My problem is why my client's receiving of this data.

PACKET_SIZE = 9216
mysocket.sendto('GO!', server_addr)
while True:
    resp = mysocket.recv(PACKET_SIZE)
    worker_thread.enqeue_packet(resp)

But by the time it gets back up to .recv(), it's missed a few udp packets (that I've confirmed are being sent using wireshark). I can fix this by making the server send slightly slower (actually, including logging statements is enough of a delay to make everything function).

How can i make sure that socket.recv doesn't miss anything in the time it takes to process a packet? I've tried pushing the data out to a separate thread that pushes it into a queue, but it's still not enough.

Any ideas? select, recv_into, setblocking?

if you are concerned with receiving every packet you shouldn't use UDP... UDP is designed for protocols that allow for dropped packets — celeriko
You already said it yourself, since you know that UDP is unreliable. There's no option that you can turn on to make it reliable. — nos
If your server sends, say, 100 packets per second, and your client can only process, say, 90 packets per second... you cannot win. Send slower, process faster, or assume that packets can be dropped. — rodrigo

Steffen Ullrich Steffen Ullrich · Accepted Answer · 2014-04-30T23:00:09

While you already know, that UDP is not reliable, you maybe missed the other advantages of TCP. Relevant for you is that TCP has flow control and automatically scales down if the receiver is unable to cope with the senders speed (e.g. packet loss). So for normal connections TCP should be preferred for data transfer. For high latency connections (satellite link) it behaves too bad in the default configuration, so that some people design there custom transfer protocols (mostly with UDP), while others just tune the existing TCP stack.

I don't know why you use UDP, but if you want to continue to use it you should add some kind of back channel to the sender to inform it from current packet loss, so that it can scale down. Maybe you should have a look at RTCP, which accompanies RTP (used for VoIP etc).

Python UDP socket misses packets

1 Answers