boost lockfree queue behavior in case fixed_sized<true>

Question

when i declare my queue as

typedef boost::lockfree::queue<MyMessage, boost::lockfree::fixed_sized<true>> MessageQueue

and i try to use the queue as a multiple-producer-single-consumer queue, i.e. post messages to it from multiple threads and poll the queue for messages from a single thread, the consumer thread gets locked. nothing moves forward.

but when i don't use the fixed size property, i.e. don't specify the template parameter fixed_sized<true> , the queue works fine. But in this case the queue actually does a memory allocation/deallocation which defeats the purpose of using a lockless queue.

So my question is "does boost::lockfree::queue becomes a single-producer-single-consumer queue if set with the property fixed_sized<true>?"

is there some different push/pop methods i need to use?

I want an MPMC queue which does not do memory allocation deallocation. if boost does not provide this, is there any other queue that i can use??

Thanks in advance,

The documentation says "The queue class provides a multi-writer/multi-reader queue", nothing in the documentation indicates this would change with a fixed_size policy. Additionally, the boost lock-free queue "uses a freelist for memory management, freed nodes are pushed to the freelist and not returned to the OS before the queue is destroyed" (per default) therefore memory allocation does not occur for every push operation even w/o fixed_size. Please provide a SSCCE. — dyp
the question above describes my observations. as soon as i use the fixed_size policy, the queue gets stuck if multiple threads call push(). It doesnt get stuck if i ensure that only a single thread calls push(). How to explain this? — weima
push is documented to be "thread-safe", it would really be nice if you could post a small code example that replicates the problem. Maybe it's even a bug in the boost version you're using. As far as I can see the documentation says it's mutli-write/multi-reader, independent from the policies. — dyp
i will not be able to share the code that i am using. but when i take the example code from boost, boost.org/doc/libs/1_54_0/doc/html/lockfree/examples.html and modify it with fixed_size<true>, the problem doesnt occur. but it does occur in my code. it goes away as soon as i remove the fixed_size parameter. i dont know how to explain this. — weima
If I interpret the docs correctly, the fixed_size<true> policy implies two changes: if you call push and no further capacity is available, push fails and returns false. Additionally, the maximum capacity that can be set is limited typically to 2**16-2 elements. You'll have to manually set the capacity via the ctor queue(size_type) or the reserve member function, the default ctor will set the capacity to 0. For a capacity of 0, every push will fail. — dyp

dyp dyp · Accepted Answer · 2013-08-07T12:59:38

So my question is does boost::lockfree::queue becomes a single-producer-single-consumer queue if set with the property fixed_size?

Answer: According to the docs, no. The member functions have different thread safety, but you can use the lockfree queue as a MRMW queue no matter if the policy is fixed_size<true> or fixed_size<false> (e.g. member functions push and pop are thread-safe).

See the comments to the OP for a discussion. Short version:

If I interpret the docs correctly, the fixed_size<true> policy implies the following changes:

If you call push and no further capacity is available, push fails and returns false.
The maximum capacity that can be set is limited typically to 2¹⁶-2 elements.
As the capacity is not changed automatically, you'll have to manually set the capacity via the ctor queue(size_type) or the reserve member function. The default ctor will set the capacity to 0. (This implies that for the default capacity of 0, every push will fail.)

boost lockfree queue behavior in case fixed_sized<true>

1 Answers