activemq, jms topics and subscribers with selectors

Question

I need some help with topics and selectors.

I have a scenario with a topic having multiple durable subscribers (each with a selector) not all messages going into the topic aren't read by the consumers - because of unmatching selectors.

This is correct behavior.

However the problem occurs when the unmatched messages reach a certain quantity threshold, because at that point no other messages are being delivered to the consumers activemq tries to dispatch those old unmatchable messages, but since there is no consumer for them everything is stuck can anybody help with this?

My setup is ActiveMq 5.5

is there some configuration option, or is it just a flawed design?

Answer 1

I'd say this is a flawed design given there are better alternatives and perhaps a bug in ActiveMQ.

First question: is your producer publishing to this topic setting the JMSExpiration header on those messages?

If yes, the first thing I'd do is create a Jira issue detailing the scenario you described above, because it does seem incorrect that ActiveMQ will continue hold on to and continue send messages for which no selectors apply.

As for flawed design, the minute you hear yourself saying "I need durable subscribers" and you are using ActiveMQ, you should immediately turn to using Virtual Destinations instead. Virtual Destinations have the advantages of topics in that a producer can send a message to a destination and have that message propagated to N number of other destinations for consumption, but doesn't have the disadvantages that come with having durable subscribers on a topic. Read more about Virtual Destinations here.

Answer 2

This is related to the way that ActiveMQ handles sparse selectors. The current implementation doesn't page into the store to look for message matching sparse selectors so you need to make some configuration changes to try and work around this. You can set the maxBrowsePageSize in the configured destination policy, the default is 400. See this page.

Virtual destinations in ActiveMQ is probably the better choice, it almost always is when thinking about using durable subscribers. You could however add some message expiration to you messages and configure the policy to expire messages on inactive durable subscribers if you use a SNAPSHOT version of ActiveMQ 5.6.

Answer 3

It does seem like a bug, (or dashedly inconvenient at the least) but there is a work around using Virtual Destinations.

Quote:

Note that making a topic virtual does add a small CPU overhead when sending messages to the topic but it is fairly small. From version 5.4, dispatch from virtual topics to subscription queues can be selectorAware such that only messages that match one of the existing subscribers are actually dispatched. Using this option prevents the build up of unmatched messages when selectors are used by exclusive consumers.