Python socket.recv with MSG_DONTWAIT

Question

I'm almost always receiving on a socket in blocking mode and this works fine. Very occasionally I don't want to wait - if there is data on the socket I want it now, otherwise I will try again later.

I thought I could do this using the flags argument to socket.recv(), but it seems not to work. I can achieve the effect I want using the socket.setblocking() and socket.settimeout() calls, but this seems clumsy.

From the python socket documentation the flags argument takes the same meanings as for Unix recv:

MSG_DONTWAIT (since Linux 2.2)
    Enables nonblocking operation; if the operation would block, the 
    call fails with the error EAGAIN or EWOULDBLOCK. This provides 
    similar behavior to setting the O_NONBLOCK flag (via the fcntl(2) 
    F_SETFL operation), but differs in that MSG_DONTWAIT is a per-call 
    option, whereas O_NONBLOCK is a setting on the open file description 
    (see open(2)), which will affect all threads in the calling process 
    and as well as other processes that hold file descriptors referring 
    to the same open file description.

I read this to mean I could pass socket.MSG_DONTWAIT to get non-blocking operation on that call only. Possibly this isn't correct - I could also read this as it would always return an error as the call in principle would be blocking. In which case, this is all irrelevant.

Some example code:

A simple blocking call. As expected, this takes about 0.5s.
MSG_DONTWAIT: I hoped would be a non-blocking call, returning very quickly. Actually this also took about 0.5s
Non-blocking by re-configuring the port. This actually takes about 50us, so it clearly isn't blocking in the way the first two calls were.

import socket
import time

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.settimeout(0.5)

starttime = time.time()
try:
    m = sock.recv(100)
except socket.timeout as e:
    pass
endtime = time.time()
print(f'sock.recv(100) took {endtime-starttime}s')  # 0.5s

starttime = time.time()
try:
    m = sock.recv(100, socket.MSG_DONTWAIT)
except socket.timeout as e:
    pass
endtime = time.time()
print(f'sock.recv(100, socket.MSG_DONTWAIT) took {endtime-starttime}s')  # 0.5s 

starttime = time.time()
timeout = sock.gettimeout()
sock.setblocking(0)
try:
    m = sock.recv(100)
except BlockingIOError as e:
    pass
sock.settimeout(timeout)
endtime = time.time()
print(f'sock.recv(100) with non-blocking set took {endtime-starttime}s')  # 4.96e-5s

Questions:

Am I just wrong about the use of MSG_DONTWAIT? Should it work in the way I am trying to use it?
Is there a better way to toggle blocking and non-blocking calls to recv()

Nikolaos Chatzis Nikolaos Chatzis · Accepted Answer · 2020-11-03T13:37:43

Regarding "1. Am I just wrong about the use of MSG_DONTWAIT? Should it work in the way I am trying to use it?":

No, you are not wrong, but there is a small issue with the way you test. Specifically, your
MSG_DONTWAIT test is for a blocking socket with a timeout of 0.5s. This is because you have sock.settimeout(0.5) before your first test (perhaps you overlooked that this affects your second test).

If I update the exception type in your MSG_DONTWAIT test (which is a another indication that the socket is blocking) and try in a "clean" session, I get what you expect you would get:

>>> import socket
>>> import time
>>> 
>>> sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
>>> starttime = time.time()
>>> try:
...     m = sock.recv(100, socket.MSG_DONTWAIT)
... except BlockingIOError as e:
...     pass
... 
>>> endtime = time.time()
>>> print(f'sock.recv(100, socket.MSG_DONTWAIT) took {endtime-starttime}s')  
sock.recv(100, socket.MSG_DONTWAIT) took 0.0007114410400390625s

If I "forget" to exclude sock.settimeout(0.5), I get a socket.timeout exception after 0.5s:

>>> import socket
>>> import time
>>> 
>>> sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
>>> sock.settimeout(0.5)  # <= see this
>>> 
>>> starttime = time.time()
>>> try:
...     m = sock.recv(100, socket.MSG_DONTWAIT)
... except socket.timeout as e:
...     pass
... 
>>> endtime = time.time()
>>> 
>>> print(f'sock.recv(100, socket.MSG_DONTWAIT) took {endtime-starttime}s')
sock.recv(100, socket.MSG_DONTWAIT) took 0.501746416091919s

Regarding "2. Is there a better way to toggle blocking and non-blocking calls to recv()": depending on the needs of your application, you may want to take a look at select (and the "Non-blocking Sockets" section in Socket Programming HOWTO and this)

Python socket.recv with MSG_DONTWAIT

1 Answers