2
votes

Is there anyway we can perform a partition aware operation on Hazelcast distributed queue?

So for example, I would have multiple consumer nodes on a queue, and would expect 'similar' type of messages to be processed by the same node everytime. By similar type, I mean some business key for the message.

Currently we are having a distributed streaming data processing ecosystem, by consuming messages from local entry listener on a IMap. A particular object model property is set as the key, so we know the models are distributed in partitions key wise. The processing logic can thus be executed locally and without using a distributed lock (or any lock at all as per design contract). I would expect the similar behaviour using a distributed blocking queue instead.

Is that feasible? Using Hazelcast 3.3.3

1

1 Answers

0
votes

You know that the queue (currently) is not a partitioned data-structure? So there is a single member in the cluster responsible for storing all data in that queue (and another member for the backup).

You can control where a queue is stored, e.g. if you have 2 queues and want them to be stored in the same partition, use

foo@somekey bar@somekey

In this case both queues are stored in the 'somekey' partition.