Do Python's queue.Queue get/put methods hold the lock for that queue while blocking?

Question

I have a conventional multithreaded producer/consumer pattern with a queue.Queue in between. This implementation of queues is thread safe. The consumer can consume the data in two ways:

Blocking

while self.running:
    data = self.receiver_q.get(block=True)
    # do something with data

Non-blocking

while self.running:
    try:
        data = self.receiver_q.get(block=False)
    except queue.Empty:
        continue
    # do something with data

In the blocking method, the consumer waits until there is data in the queue. During this time, is the consumer holding the lock on the queue? I can't imagine a way for it to hold the lock while allowing the queue to have new data placed on it.

Also, is there a performance difference between the two patterns?

Prune Prune · Accepted Answer · 2018-03-22T21:20:16

(1) No, the consumer is not holding a lock; it's merely blocked until the request is satisfied, but the queue is still available. If there are multiple consumers, then this one is in the pool of potential recipients; the resolution function will choose the order of satisfaction.

(2) Any performance difference depends on the implementation. Don't ask us -- you have the foremost authority in front of you: your computer. Use a test scenario of your choice, and the timeit facility.

Do Python's queue.Queue get/put methods hold the lock for that queue while blocking?

1 Answers