Priority Blocking Queue in Hazelcast

1
votes

I am evaluating Hazelcast IMDG with a view to use it to reduce some database bottle necks. Our application runs on a number of nodes processing data in parallel from a database sitting in the middle.

We use a semaphore table to distribute the work and this is what we are considering to replace with a Hazelcast blocking queue.

However some messages to process has a higher priority than others and this where I started having doubts as it seems that Hazelcast does not have an equivalent of java.util.concurrent.PriorityBlockingQueue.

My first question is whether there is a way to achieve this with Hazelcast. My second question is whether there are other IMDG data grids easy to configure and embed into an existing application that would allow this.

Thank you in advance for your inputs.

2
javahazelcast
Hazelcast doesn't provide this currently, you have to built it yourself. But it's not difficult if you use the programmer's SPI. PM me if you need details – Neil Stevenson
Not sure how to PM you. But why not provide here your solution/ideas so everyone who is interested will benefit from it. I have some ideas myself about wrapping an IMap<Priority, IQueue<Message>> but being new to IMDG I would prefer some expertise from someone with more experience. – Julian
Good idea. Gimme a few days to write it. – Neil Stevenson
Not forgotten... – Neil Stevenson
Thanks was wondering if you did indeed :-). I tried my own version myself but run in all kind of problems. Obviously my knowledge is not mature enough. In the meantime I started reading Apache Ignite documentation as their in memory tables may easier help me with my problem but I admit so far I like Hazelcast most. Anyway I'll see depending on what Ideas you come up with – Julian

2 Answers

2
votes

I know it's a bit late but Hazelcast finally offers an out of the box support for distributed PriorityQueue. See: https://github.com/hazelcast/hazelcast/pull/17201

It's an extension of the regular queue, you just add a comparator to it in the queue configuration (https://github.com/hazelcast/hazelcast/pull/17201/files#diff-adec01c0eef8a0df650ac3c02789b91a2eafc8c2ffbb6e77c563dc88f5a5e9a2R559):

<priority-comparator-class-name>your.comparator.Class</priority-comparator-class-name>

I will update this post once the Hazelcast IMDG 4.1 is out and the documentation is added.

1
votes

Does this https://github.com/hazelcast/hazelcast-code-samples/tree/master/spi/priority-queue do what you need ?

It's a fully working example -- shows how Hazelcast can be extended to add the priority queue. Needs a little tweaking to make it production quality though, and some unit tests wouldn't go amiss.