What's the correct way to perform multiple operations on a threading queue?

Question

I need to perform a set of operations on a queue.Queue, specifically to go through it and pack duplicate contiguous elements. I also want to be sure there's only one thread accessing the queue during this set of operations. I see that the Queue object has a mutex, but I also assume it's for the queue to perform locking during its own operations. Having another mutex does not guarantee that the queue is not accessed by another thread in the meantime, unless I organize my code to lock the second mutex.

Is there a trick I am not aware of in order to perform these operations safely?

Answer 1

Use the Queue object's mutex for this purpose. It's there precisely to synchronize modifications to the underlying queue.

Perhaps subclass and add a method (like the one in https://stackoverflow.com/a/7641976/132382) to pack contiguous duplicates in self.queue?

Alternatively and I think better, subclass queue.Queue and override _put so that contiguous duplicates are handled as soon as they are added to the queue. Invocations of _put are already mutex-protected (and _put, _get, etc. are how the standard subclasses like LifoQueue are implemented):

class PackedQueue(queue.Queue):

  def _put(self, item):
    if item != self.queue[-1]:
      self.queue.append(item)